neo4j using example

Our project is a360.ru. The main goal of our task is to find many paths from one map point to another map point ( map points are cities, railway stations, airports, showplaces, etc. ).  We use neo4j-server for transport db storage (graph of the railroad network, flights, etc.). We use neo4j plugin for rapid search routes between two point (cities/stations/airports). Our search include Shortest paths algorithms and Dijkstra algorithms for set of tasks (we use different weights for search different routes).

Create neo4j plugin

We use neo4j for transport db storage and for find routes. Find routes process is large task, and so we want find routes via server plugin.

It's some step to create plugin for neo4j-server:
1) The plugin class must like as follows:

/**
 * License information
 */

// package name - for keeping structure and your convenience
package ru.companyName.neo4j.plugins;

// import classes from neo4j
import org.neo4j.graphdb.GraphDatabaseService;
...

// import standard java classes
import java.util.ArrayList;
...

// decorator description of plugin (plugin class)
@Description( "An extension to the Neo4j Server for find routes between two nodes" )
// your plugin class must extension ServerPlugin class
public class MyPlugin extends ServerPlugin {
// follow - standard structure of the class (there is implementation of your algorithms)
Set<String> trueTypes;

// create class method
public TransportRouter() {
this.trueTypes = new HashSet<String>(Arrays.asList("type1", "type2"));
}

// http-query method
@Name( "get_paths" )
@Description( "Get paths from node to node" )
// from this class you will call your plugin class
@PluginTarget( GraphDatabaseService.class )
public Representation getPaths(
@Source GraphDatabaseService graphDb,
     @Description( "The id of the node from" )
                @Parameter( name = "id_from" ) Node nodeFrom,
     @Description( "The id of the node to" )
                @Parameter( name = "id_to" ) Node nodeTo
                ) throws Exception
{
                // implementation of your algorithms
                ...
                // returnable type Representation allow type String (in it we put string in json; also plugin method can return Iterable<Node> and Iterable<Relationship>)
return ValueRepresentation.string(result.toString());
}
}

2) Then plugin class must be compiled (Eclipse, javac, etc.), and then all compiled classes (*.class) put into JAR.
Directories tree in JAR-archive must like as name of package:
/ru/companyName/neo4j/plugins/<there are compiled classes>
JAR also must contain file:
/META-INF/services/org.neo4j.server.plugins.ServerPlugin
In this file you must write name of your package: ru.companyName.neo4j.plugins.TransportRouter

3) put resulting JAR in folder .../neo4j-server/plugins
4) restart server
5) get plugin list in server (in bash):
curl http://localhost:7474/db/data/ext/
6) Then we can query our plugin class method:
curl -X POST http://localhost:7474/db/data/ext/MyPlugin/graphdb/get_path -H "Content-Type: application/json" -d '{"id_from": "http://localhost:7474/db/data/node/1", "id_to": "http://localhost:7474/db/data/node/2"}'
In this string after plugin name ("MyPlugin") we write "graphdb", because in decorator @PluginTarget( GraphDatabaseService.class ) we difine this class as parent class. Parameter -d define dictionary (Map), which consider all input parameters for our plugin method (which define via @Parameter).

There is problem in process of test plugin: DB wasn't see my plugin. This problem may occur by some reason (if your compilation classes was access):

- there isn't file /META-INF/services/org.neo4j.server.plugins.ServerPlugin or wrong string in it;

- there is error while initialization plugin class. You can try to create you class from other application to see this errors.

If your plugin was contained in the list of plugins, you may try query class method (from bash). And you will see all errors in the tail.

mysql: add autoincrement column

alter table `new_transport_lines` add column `id` int not null auto_increment KEY FIRST

Could not open a connection to your authentication agent

from here: http://funkaoshi.com/blog/could-not-open-a-connection-to-your-authentication-agent

SSH private-keys are usually stored encrypted on the computers they are stored on. A pass-phrase is used to decrypt them when they are to be used. Since most people use SSH public-private key-pairs to get around typing in passwords all the time, the ssh-agent daemon exists to store decrypted private-keys you plan on using in a given session. The thing most people get tripped up on when using ssh-agent is that what the program outputs, some borne or csh shell commands, needs to be run. It may look like ssh-agent has set some variables for you, but it has in fact done no such thing. If you call ssh-add without processing ssh-agent’s output, it will complain it is unable to open a connection to your authentication agent. The most straightforward way to run ssh-agent on the command line is as follows: eval `ssh-agent`. After doing this, calls to ssh-add should succeed without error.

mysql: Got a packet bigger than 'max_allowed_packet' bytes

Mysql выдаёт сообщение при попытке залить дамп: "Got a packet bigger than 'max_allowed_packet' bytes"

Открываем mysql ещё в одной консоли, пишем:set max_allowed_packet=100M

Далее повторяем попытку импорта дампа - работает!

Возможно, ещё пригодится параметр: net_buffer_length=1000000

about time in Python

http://asvetlov.blogspot.com/2011/02/date-and-time.html

fabric Remote via SSH

@hosts(EC2_MYSQL_HOST)
def ecMysqlUp():
    env.key_filename = EC2_KEY_FILENAME
    env.user = 'ubuntu'
    print env.host