8/07/2017

Tired of seeing that 500 Bad gateway error while deploying a Springboot application in AWS...?

By default, Spring Boot applications will listen on port 8080. Elastic Beanstalk assumes that the application will listen on port 5000. There are two ways to fix this discrepancy: change the port Elastic Beanstalk is configured to use, or change the port the Spring Boot application listens on. For this post, we will change the port the Spring Boot application listens on.
The easiest way to do this is to specify the SERVER_PORT environment variable in the Elastic Beanstalk environment and set the value to 5000. (The configuration property name is server.port, but Spring Boot allows you to specify a more environment variable-friendly name).
On the Configuration page in your environment, under Software Configuration, click the settings icon.

AWS Console

8/04/2014

Java 8 - Default Methods

 Before Java 8, it was virtually impossible to change an interface once published. Any change e.g. addition of a new method would have broken all clients.  Since backward compatibility is top priority for Java engineers, and it wasn't practical to break all clients, they came up with idea of default method.  

Java 8 enables us to add non-abstract method implementations to interfaces by utilizing the default keyword. This feature is also known as Extension Methods. Here is an example:

interface Car{
    int drive(int speed, int direction);
      
    default int wheels(){
        return 4;
    }
}

Besides the abstract method drive() the interface Caralso defines the default method wheels(). Any concrete classes of Carinterface only have to implement the abstract method drive(). The default method wheels() method can be used directly.

  Car myCar = new Car(){
      @Override
      public int drive(int speed, int direction){
          return x*y;
      }
  };

int totalWheels = myCar.wheels();//Invokes the default implementation in the interface.

1/28/2014

Fast boot the Android emulator in less than 15s

The default emulator is annoyingly slow. Read on this post to boot your emulator in 15s

1) You HAVE to have an Intel CPU, anything else (AMD) will NOT work. Too bad AMD users ;)

2) OPEN adt-bundle-xxx/SDK Manager.exe
image
3) Checkbox extras/Intel x86 Emulator Accelerator (HAXM)

image

4) Also checkbox Android 4.4.2 (API 19)/Intel x86 Atom System Image

image
5) INSTALL ... wait...

6) Upon completion, browse here:
 \adt-bundle-xxx\sdk\extras\intel\Hardware_Accelerated_Execution_Manager\IntelHaxm.exe and install the driver

7) Reset your computer (not sure if this is necessary, but do it anyway)

8) Finally start Eclipse >> Window >> Android VD Manager

9) Copy my settings; you can edit them later. As long as you ensure your CPU is intel atom, you're good. You can FURTHER speed up emulator by checking Use Host GPU. ALSO further speed up by using a lower resolution, like 800x600.

image

8/07/2013

Examples of GOF Design Patterns


Original @Author BalusC. Re-post from Stackoverflow.

Creational patterns

Abstract factory (recognizeable by creational methods returning the factory itself which in turn can be used to create another abstract/interface type)

Builder (recognizeable by creational methods returning the instance itself)

Factory method (recognizeable by creational methods returning an implementation of an abstract/interface type)

Prototype (recognizeable by creational methods returning a different instance of itself with the same properties)

Singleton (recognizeable by creational methods returning the same instance (usually of itself) everytime)


Structural patterns

Adapter (recognizeable by creational methods taking an instance of different abstract/interface type and returning an implementation of own/another abstract/interface type which decorates/overrides the given instance)

Bridge (recognizeable by creational methods taking an instance of different abstract/interface type and returning an implementation of own abstract/interface type which delegates/uses the given instance)

  • None comes to mind yet. A fictive example would be new LinkedHashMap(LinkedHashSet, List) which returns an unmodifiable linked map which doesn't clone the items, but uses them. The java.util.Collections#newSetFromMap() and singletonXXX() methods however comes close.

Composite (recognizeable by behavioral methods taking an instance of same abstract/interface type into a tree structure)

Decorator (recognizeable by creational methods taking an instance of same abstract/interface type which adds additional behaviour)

Facade (recognizeable by behavioral methods which internally uses instances of different independent abstract/interface types)

Flyweight (recognizeable by creational methods returning a cached instance, a bit the "multiton" idea)

Proxy (recognizeable by creational methods which returns an implementation of given abstract/interface type which in turn delegates/uses a different implementation of given abstract/interface type)

The Wikipedia example is IMHO a bit poor, lazy loading has actually completely nothing to do with the proxy pattern at all.

Behavioral patterns

Chain of responsibility (recognizeable by behavioral methods which (indirectly) invokes the same method in another implementation of same abstract/interface type in a queue)

Command (recognizeable by behavioral methods in an abstract/interface type which invokes a method in an implementation of a different abstract/interface type which has been encapsulated by the command implementation during its creation)

Interpreter (recognizeable by behavioral methods returning a structurally different instance/type of the given instance/type; note that parsing/formatting is not part of the pattern, determining the pattern and how to apply it is)

Iterator (recognizeable by behavioral methods sequentially returning instances of a different type from a queue)

Mediator (recognizeable by behavioral methods taking an instance of different abstract/interface type (usually using the command pattern) which delegates/uses the given instance)

Memento (recognizeable by behavioral methods which internally changes the state of the whole instance)

Observer (or Publish/Subscribe) (recognizeable by behavioral methods which invokes a method on an instance of another abstract/interface type, depending on own state)

State (recognizeable by behavioral methods which changes its behaviour depending on the instance's state which can be controlled externally)

Strategy (recognizeable by behavioral methods in an abstract/interface type which invokes a method in an implementation of a different abstract/interface type which has been passed-in as method argument into the strategy implementation)

Template method (recognizeable by behavioral methods which already have a "default" behaviour definied by an abstract type)

Visitor (recognizeable by two different abstract/interface types which has methods definied which takes each the other abstract/interface type; the one actually calls the method of the other and the other executes the desired strategy on it)

8/06/2013

NoSQL databases

NoSQL databases have been getting lot of attention over the last few years for their performance, scalability, schema flexibility and analytics capabilities. While relational databases are still good choice for certain use cases - like structured data and applications that require ACID transactions - NoSQL databases are better suited for use cases where:
  •     The data stored is semi-structured or unstructured in nature
  •     The applications that access this data require a certain level of performance and scalability
  •     The applications that access this data are ok with eventual consistency
 
Non-relational databases typically support the following capabilities:
  •     Schema flexibility
  •     Shared nothing architecture
  •     Sharing as part of the data storage model
  •     Asynchronous replication
 
Few NoSQL databases trending in the market:
If you are using or have used any of these databases, please share the details on how it is\was used.
 
Document Databases
  • MongoDB: MongoDB is an open-source document oriented database.
  • CouchDB: Apache CouchDB is a database that uses JSON for documents, JavaScript for MapReduce queries, and HTTP for an API.
  • RavenDB: RavenDB is a document-oriented database based on .NET language.
 
Graph Databases
  • Neo4j: Neo4j is a property graph database; supports ACID transactions.
  • InfiniteGraph: Graph database used to persist and traverse relationships between objects, supports distribute data stores.
  • AllegroGraph: AllegroGraph is a graph database that uses memory utilization in combination with disk-based storage for scalability, supports SPARQL, RDFS++, and Prolog reasoning.
 
Key Value Data Stores
  • Riak: Riak is an open source, distributed key value database, supports data replication and fault-tolerance.
  • Redis: Redis is an open source key-value store. Supports master-slave replication, transactions, Pub/Sub, Lua scripting, Keys with a limited time-to-live.
  • Dynamo: Dynamo is a key-value distributed data store. It is directly implemented as Amazon DynamoDB; used in Amazon S3 product.
  • Voldemort: Distributed key-value storage system with the data replication and partitioning.
  • Aerospike: Aerospike database is a key-value store; supports hybrid memory architecture and data integrity with strong or tunable consistency.
 
Columnar Databases
  • Cassandra: Cassandra is column database that supports data replication across multiple data centers. Its data model offers column indexes, log-structured updates, support for denormalization, materialized views, and built-in caching.
  • HBase: Apache HBase is an open-source, distributed, versioned, column-oriented store modeled after Google's Bigtable. It provides Bigtable-like capabilities on top of Hadoop and HDFS.
  • Amazon SimpleDB: Amazon SimpleDB is a non-relational data store that offloads the work of database administration. Developers store and query data items using web services requests.
  • Apache Accumulo: Apache Accumulo sorted, distributed key/value data store created based on Google's BigTable design and is built on top of Apache Hadoop, Zookeeper, and Thrift technologies.
  • Hypertable: Hypertable is an open source, scalable database, also modeled after Bigtable; supports sharing.
  • Azure Tables: Windows Azure Table Storage Service offers NoSQL capabilities for applications that require storage of large amounts of unstructured data. Tables can auto-scale to store up to several terabytes of data. They are accessible via REST and managed APIs.

lambdaj - Express Business Rules in Psuedo English Syntax

I came across this framework  called  'lambdaj' when I was looking for ways to manipulate a collection without writing loops.

lambdaj is a library that makes easier to address this issue by allowing to manipulate collections in a pseudo-functional and statically typed way. With lot of few sugar methods lambdaj gives the ability to express the business rules of your application in pseudo-english syntax. I remember reading somewhere these are part of Java 8 bundle already!!!!

This is a proven, simple and eaze to use library. If there are similar frameworks, let me know.

Sample Code:
TestLambdaj.java

package com.scjp;

import static ch.lambdaj.Lambda.forEach;
import static ch.lambdaj.Lambda.joinFrom;
import static ch.lambdaj.Lambda.maxFrom;
import static ch.lambdaj.Lambda.min;
import static ch.lambdaj.Lambda.on;
import static ch.lambdaj.Lambda.sort;
import java.util.Arrays;
import java.util.List;

public class TestLambdaj {

    public static void main(String[] args) {
        Person p1 = new Person("Harry", 50);
        Person p2 = new Person("Dave", 70);
        Person p3 = new Person("Wilson", 30);
        Person p4 = new Person("Mark", 20);

        List persons = Arrays.asList(p1, p2, p3, p4);
        // Sort the collection by Age
        List sortedAge = sort(persons, on(Person.class).getAge());
        System.out.println(sortedAge);   //Output: [Mark - 20, Wilson - 30, Harry - 50, Dave - 70]


        // Sort the collection by Name
        List sortedName = sort(persons, on(Person.class).getName());
        System.out.println(sortedName); //Output:[Dave - 70, Harry - 50, Mark - 20, Wilson - 30]

        // Find the max Age in the collection
        float maxAge = maxFrom(persons).getAge();
        System.out.println(maxAge);//Output:70.0

        // Find the min Age in the collection
        float minAge = min(persons, on(Person.class).getAge());
        System.out.println(minAge); //Output:20.0

        // Print all the names of the person
        String names = joinFrom(persons).getName();
        System.out.println(names); //Output:Harry, Dave, Wilson, Mark

        // Set the age of all elements inside the collection to 20
        forEach(persons).setAge(20);
        System.out.println(p1.getAge()); //Output:20
    }

}



Person.java
package com.scjp;

public class Person {

    private String name;
    private int age;

    Person(String name, int age) {
        this.name = name;
        this.age = age;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String toString() {
        return this.name + " - " + this.age;
    }

}

8/01/2013

Mahout - Non Hadoop implementation for Collaborative Filtering

It's been a week with some serious learning stuff. I stumbled upon this technique called 'Collaborative Filtering' and found it very interesting. Though the concept is been there for ages, time and again there has been new implementations around it. With little googling I managed to run a stand alone program (Non-hadoop) using Apache Mahout. For those it is new, read on.

Collaborative Filtering, is a technique of predicting preferences given the preferences of others in the group. There are two main approaches - user based and item based. In case of user-based filtering, the objective is to look for users similar to the given user, then use the ratings from these similar users to predict a preference for the given user. In case of item-based recommendation, similarities between pairs of items are computed, then preferences predicted for the given user using a combination of the user's current item preferences and the similarity matrix. Example shown below uses user's based receommendation (think of something Amazon's recommendations)

Inputs...
The input to such a system is either a 3-tuple of (UserID, ItemID, Rating) or a 2-tuple of (UserID, ItemID).  In Mahout, this input is represented by a DataModel class, which can be created from a file of these tuples (either CSV or TSV), one per line. Other ways to populate the DataModel also exist, for example, programatically or from database.

Under the hoods...
A user-based Recommender is built out of a DataModel, a UserNeighborhood and a UserSimilarity. A UserNeighborhood defines the concept of a group of users similar to the current user - the two available implementations are Nearest and Threshold. The nearest neighborhood consists of the nearest N users for the given user, where nearness is defined by the similarity implementation. The threshold neighborhood consists of users who are at least as similar to the given user as defined by the similarity implementation. The UserSimilarity defines the similarity between two users - implementations include EuclideanDistance, Pearson Correlation, Uncentered Cosine, Caching, City Block, Dummy, Generic User, Log Likelihood, Spearman Correlation and Tanimoto Coefficient similarity.

Input File:
There are five users with the matrix (user,item,rating). The sample program shown below will recommend items for other users based on the below list.

1,101,5.0
1,102,3.0
1,103,2.5

2,101,2.0
2,102,2.5
2,103,5.0
2,104,2.0

3,101,2.5
3,104,4.0
3,105,4.5
3,107,5.0

4,101,5.0
4,103,3.0
4,104,4.5
4,106,4.0

5,101,4.0
5,102,3.0
5,103,2.0
5,104,4.0
5,105,3.5
5,106,4.0

Sample Code - Java Implementation:
package com.cf;

import java.io.File;
import java.util.List;

import org.apache.mahout.cf.taste.impl.model.file.FileDataModel;
import org.apache.mahout.cf.taste.impl.neighborhood.NearestNUserNeighborhood;
import org.apache.mahout.cf.taste.impl.recommender.GenericUserBasedRecommender;
import org.apache.mahout.cf.taste.impl.similarity.PearsonCorrelationSimilarity;
import org.apache.mahout.cf.taste.model.DataModel;
import org.apache.mahout.cf.taste.neighborhood.UserNeighborhood;
import org.apache.mahout.cf.taste.recommender.RecommendedItem;
import org.apache.mahout.cf.taste.recommender.Recommender;
import org.apache.mahout.cf.taste.similarity.UserSimilarity;

public class TestClass {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        // TODO Auto-generated method stubs

        File file = new File("C:\\Users\\Kaveesh\\workspace\\Collaborative_Filtering\\ds\\test.txt");
        DataModel model = new FileDataModel(file);

        UserSimilarity similarity = new PearsonCorrelationSimilarity(model);
        UserNeighborhood neighborhood = new NearestNUserNeighborhood(2,
                similarity, model);

        Recommender recommender = new GenericUserBasedRecommender(model,
                neighborhood, similarity);

        //Now we can get 2 recommendations for user ID "1"
        List recommendations = recommender.recommend(1, 2);

        for (RecommendedItem recommendation : recommendations) {
            System.out.println(recommendation);
        }

    }

}

Output:
RecommendedItem[item:104, value:4.257081]
RecommendedItem[item:106, value:4.0]


Jars Needed:
 mahout-core-0.8-job.jar
 mahout-core-0.8.jar
 mahout-integration-0.8.jar
 mahout-math-0.8.jar
 slf4j-api-1.7.5.jar
 slf4j-jcl-1.7.5.jar


 Ah..sounds simple? Not really. Now the bigger picture. Think of Amazon\Netflix running their code on Mahout and Hadoop. Personalized recommendations are provided for each online users scanning miliion of users and millions of items. Needless to say, the results of the recommendations are serviced less than a sec.

So my next step is to create a database in MySQL and integrate with Mahout and Hadoop using MySQL-Hadoop-Applier.

10/09/2012

Running Hadoop MapReduce on Cygwin with proper credentials




I just kept getting the below exception while trying to run a simple map reduce program on windows(XP).
java.io.IOException: Failed to set permissions of path: \tmp\dinesh\mapred\staging\dinesh-1143336710\.staging to 0700

Nothing worked till I saw a simple workaround for the problem. Download the jar from the below link and follow the instructions. 

Download here

Tired of seeing that 500 Bad gateway error while deploying a Springboot application in AWS...?

By default, Spring Boot applications will listen on port 8080. Elastic Beanstalk assumes that the application will listen on port 5000. Th...