Tuesday

Book: High performance in-memory computing with Apache Ignite has been published

The book "High performance in-memory computing with apache Ignite" has been released and
available at http://leanpub.com/ignite

Print copy of the book is available at Lulu.com & Amazon bookstore.

Support independent publishing: Buy this book on Lulu.
UP1: NOW the book is available for purchase from the Russian federation through PayPal (Ignore the yellow warning).


The goal of the book is to provide a guide for those who really need to implement the In-memory platform in their projects. At the same time, the idea behind the book is not writing a manual.

This book wraps all the topics like in-memory data grid, highly available service grid, streaming and in-memory computing use cases from high-performance computing to get the performance gain. The book will be particularly useful for those, who have the following use cases:

  • You have database bottleneck in your application and want to solve the problem.
  • You have a high volume of ACID transactions in your system.
  • You want to develop and deploy microservices in distributed fashion.
  • You have existing Hadoop ecosystem (OLAP) and want to improve the performance of the Map/Reduce jobs without making any changes in your existing Map/Reduce jobs.
  • You want to share Spark RDD directly in-memory (without storing the state to disk), which can dramatically increase the performance of the Spark jobs.
  • You are planning to migrate to microservices and the web session clustering is the problem for you.
  • You are planning to process continuous never-ending streams and complex events of data in scalable and fault-tolerant fashion.
  • You want to use distributed computations in parallel fashion to gain high performance, low latency, and linear scalability.
  • You heard about Off-heap memory but don't know how to use it in your application.
For every topic, a complete application is delivered, which will help the audience to quick start with the topic. The book is a project-based guide, where each chapter focuses on the complete implementation of a real-world scenario, the commonly occurring challenges in each scenario has also discussed, along with tips and tricks and best practices on how to overcome them. Every chapter is independent and a complete project.

Who is this book for

Target audience of this book will be IT architect, team leaders, a programmer with minimum programming knowledge, who want to get the maximum performance from their applications.

No excessive knowledge is required, though it would be good to be familiar with JAVA and Spring framework. The book is also useful for any reader, who already familiar with Oracle Coherence, Hazelcast, Infinispan or memcached.

See the full table of contents of the book here.

Happy Reading.

Sunday

Apache Ignite with Spring Data

Spring Data provides a unified and easy way to access the different kinds of persistence store, both relational database systems, and NoSQL data stores. It is on top of JPA, adding another layer of abstraction and defining a standard-based design to support persistence Layer in a Spring context.
Apache Ignite IgniteRepository implements Spring Data CrudRepository interface and extends basic capabilities of the CrudRepository, which in turns supports:
  1. Basic CRUD operations on a repository for a specific type.
  2. Access to the Apache Ignite SQL grid via Spring Data API.
With Spring Data's repositories, you only need to write an interface with finder methods to query the objects. All the CRUD method for manipulating the objects will be delivered automatically. As an example:


@RepositoryConfig(cacheName = "DogCache")
public interface DogRepository extends IgniteRepository<Dog, Long> {
    List<Dog> getDogByName(String name);
    Dog getDogById (Long id);
}
In this article, we are going to cover the following topics:
  • Create a Maven project from the scratch for using Spring Data with Apache Ignite Grid.
  • Persisting a few entities into Ignite caches through Spring Data framework.
Before we start, let's cover the prerequisites of the project in your sandbox:
  1. Java JDK 1.8
  2. Ignite version2.0
  3. Apache Maven version >3.0.3
Step 1
Let’s set up the sandbox first. Create a Maven project or Clone the project from the GitHub repository.
mvn archetype:create -DgroupId=com.blu.imdg -DartifactId=spring-data

Step 2

Modify the pom.xml, add the following maven dependencies:
<dependency>
    <groupId>org.apache.ignite</groupId>
    <artifactId>ignite-core</artifactId>
    <version>2.0.0</version>
</dependency>
<dependency>
    <groupId>org.apache.ignite</groupId>
    <artifactId>ignite-spring</artifactId>
    <version>2.0.0</version>
</dependency>
<dependency>
    <groupId>org.apache.ignite</groupId>
    <artifactId>ignite-spring-data</artifactId>
    <version>2.0.0</version>
</dependency>
<dependency>
    <groupId>org.apache.ignite</groupId>
    <artifactId>ignite-indexing</artifactId>
    <version>2.0.0</version>
</dependency>
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <version>1.4.195</version>
</dependency>
Note that, maven h2 dependency is optional. If you are getting an error like "org.h2.result.RowFactory", add the dependency explicitly.

The Domain Model

Our example domain model consisted of two different entities: Breed and Dog.

The association between Breed and Dog is ManyToOne. One Dog can have only one breed.

Step 3

Now, let’s map the domain model by creating the Java classes and annotating them with the required meta-information. Let’s start with the Breed class.
package com.blu.imdg.model;

import org.apache.ignite.cache.query.annotations.QuerySqlField;

import java.io.Serializable;

public class Breed implements Serializable {

    @QuerySqlField(index = true)
    private Long id;

    @QuerySqlField(index = true)
    private String name;

    public Long getId() {

        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "Breed{" +
                "id='" + id + '\'' +
                ", name='" + name + '\'' +
                '}';
    }
}
Note that, @QuerySqlField annotation enables the fields for SQL queries.
Create another class named Dog and add the following contents to it.

package com.blu.imdg.model;

import org.apache.ignite.cache.query.annotations.QuerySqlField;

import java.io.Serializable;
import java.sql.Date;

public class Dog implements Serializable {

    @QuerySqlField(index = true)
    private Long id;
    @QuerySqlField(index = true)
    private String name;
    @QuerySqlField(index = true)
    private Long breedid;
    @QuerySqlField(index = true)
    private Date birthdate;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public Long getBreedid() {
        return breedid;
    }

    public void setBreedid(Long breedid) {
        this.breedid = breedid;
    }

    public Date getBirthdate() {
        return birthdate;
    }

    public void setBirthdate(Date birthdate) {
        this.birthdate = birthdate;
    }

    @Override
    public String toString() {
        return "Dog{" +
                "id=" + id +
                ", name='" + name + '\'' +
                ", breedid=" + breedid +
                ", birthdate=" + birthdate +
                '}';
    }
}

Step 4

Now, lets create the Spring repository for all the pojo's created before.
package com.blu.imdg.repositories;

import com.blu.imdg.model.Dog;
import org.apache.ignite.springdata.repository.IgniteRepository;
import org.apache.ignite.springdata.repository.config.RepositoryConfig;

import java.util.List;

@RepositoryConfig(cacheName = "DogCache")
public interface DogRepository extends IgniteRepository<Dog, Long> {
    List<Dog> getDogByName(String name);
    Dog getDogById (Long id);
}
@RepositoryConfig annotation should be specified to map a repository to a distributed cache. Also, we have two finder methods getDogByName and getDogById for querying the cache.
Lets' add a similar repository for the Breed domain as follows:
package com.blu.imdg.repositories;

import com.blu.imdg.model.Breed;
import org.apache.ignite.springdata.repository.IgniteRepository;
import org.apache.ignite.springdata.repository.config.Query;
import org.apache.ignite.springdata.repository.config.RepositoryConfig;
import org.springframework.data.domain.Pageable;

import java.util.List;

@RepositoryConfig(cacheName = "BreedCache")
public interface BreedRepository extends IgniteRepository<Breed, Long> {

    List<Breed> getAllBreedsByName (String name);

    @Query("SELECT id FROM Breed WHERE id = ?")
    List<Long> getById (long id, Pageable pageable);
}

In the above BreedRepository interface, we also use @Query(queryString) annotation, which can be used if a concrete SQL query needs to be executed as a result of a method call.
Step 5
Let’s create the cache configuration class. Create an Ignite cache configuration class and mark the application configuration with @EnableIgniteRepositories annotation, as shown below:
package com.blu.imdg.repositories;

import com.blu.imdg.model.Breed;
import com.blu.imdg.model.Dog;
import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.springdata.repository.config.EnableIgniteRepositories;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableIgniteRepositories
public class SpringAppConfig {
    @Bean
    public Ignite igniteInstance() {
        IgniteConfiguration cfg = new IgniteConfiguration();
        // Setting some custom name for the node.
        cfg.setIgniteInstanceName("springDataNode");
        // Enabling peer-class loading feature.
        cfg.setPeerClassLoadingEnabled(true);
        // Defining and creating a new cache to be used by Ignite Spring Data
        // repository.
        CacheConfiguration ccfgDog = new CacheConfiguration("DogCache");
        CacheConfiguration ccfgBreed = new CacheConfiguration("BreedCache");
        // Setting SQL schema for the cache.
        ccfgBreed.setIndexedTypes(Long.class, Breed.class);
        ccfgDog.setIndexedTypes(Long.class, Dog.class);

        cfg.setCacheConfiguration(new CacheConfiguration[]{ccfgDog, ccfgBreed});

        return Ignition.start(cfg);
    }
}
Note that, we have used two separate CacheConfiguration for Breed and Dog cache. Also, set the SQL schema for the cache.

Step 6

Once all the configurations and the repositories are ready to be used, we only need to register the configuration in a Spring application context.
package com.blu.imdg;

import com.blu.imdg.model.Breed;
import com.blu.imdg.model.Dog;
import com.blu.imdg.repositories.BreedRepository;
import com.blu.imdg.repositories.DogRepository;
import com.blu.imdg.repositories.SpringAppConfig;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;

import java.sql.Date;
import java.util.List;

/**
 * Hello world!
 *
 */
public class App 
{
    private static AnnotationConfigApplicationContext ctx;
    private static BreedRepository breedRepository;
    private static DogRepository dogRepository;

    public static void main( String[] args )
    {
        System.out.println( "Spring Data Example!" );
        ctx = new AnnotationConfigApplicationContext();
        ctx.register(SpringAppConfig.class);
        ctx.refresh();

        breedRepository = ctx.getBean(BreedRepository.class);
        dogRepository = ctx.getBean(DogRepository.class);

        //fill the repository with data and Save
        Breed collie = new Breed();
        collie.setId(1L);
        collie.setName("collie");
        //save Breed with name collie
        breedRepository.save(1L, collie);

        System.out.println("Add one breed in the repository!");
        // Query the breed
        List<Breed> getAllBreeds = breedRepository.getAllBreedsByName("collie");

        for(Breed breed : getAllBreeds){
            System.out.println("Breed:" + breed);
        }
        //Add some dogs
        Dog dina = new Dog();
        dina.setName("dina");
        dina.setId(1L);
        dina.setBreedid(1L);
        dina.setBirthdate(new Date(System.currentTimeMillis()));
        //Save Dina
        dogRepository.save(2L,dina);
        System.out.println("Dog dina save into the cache!");
        //Query the Dog Dina
        List<Dog> dogs = dogRepository.getDogByName("dina");
        for(Dog dog : dogs){
            System.out.println("Dog:"+ dog);
        }

    }
}
The above code snippet is very straight forward. First, we create a Spring annotated context and register our repositories. Next, we get the reference to our BreedRepository and DogRepository to insert a few data. To query the data we use basic CRUD operations or methods that will be automatically turned into Apache Ignite SQL queries:


List<Dog> dogs = dogRepository.getDogByName("dina");
for(Dog dog : dogs){
  System.out.println("Dog:"+ dog);
}

Step 7

Let’s build and run the application. Execute the following command.
mvn clean install
mvn exec:java -Dexec.mainClass=com.blu.imdg.App

You should find a lot of log messages into the console.


The log messages confirm that two entries (dina and breed-collie) have been flushed into the Ignite cache and retrieved the dog Dina from the cache. Let’s explore the cache through Ignite Visor.

Two different caches have been created for the entities: Breed and Dog. If we scan the cache entries of the Dog cache, we should find the following entity on it.

Entity Dina has been persisted into the cache with the key of the Breed collie.
If you want to learn more about Apache Ignite (using JPA, Hibernate or MyBatis), please refer the book High Performance in-memory computing with Apache Ignite.