Looking Deeper at the Platform

• Neo4j Desktop is developers’ mission control console for all things Neo4j
• Neo4j Native Graph Database supports transactional applications and graph analytics
• Neo4j Graph Analytics helps data scientists gain new perspectives on data
• Data integration tools expedite distilling RDBMS data and other big data into graphs
• Graph visualization and discovery help communicate graph benefits throughout the organization
• The platform-wide enterprise architecture supports corporate availability, scaling and security needs

Introducing Neo4j Desktop

• Built-in user management, user security, Kerberos authentication and LDAP integration
• Performance boost from compiled Cypher, enterprise lock management and space reuse
• Schema features like Node Keys, existence constraints and composite indexes
• Scaling features like unlimited nodes and relationships, and supported Bolt language drivers
• All platform components and interfaces like query management and the Neo4j Browser
• Exposure to production deployment features like high availability, disaster recovery, secure Causal Clustering, IPv6 and least-connected load balancing.

Now Shipping: Neo4j Graph Database 3.3

• Native indexes for numeric properties, improves inserts and updates speeds dramatically.
• The bulk data importer uses 40% less memory, and adds the ability to use the page cache when memory is full.
• In clusters, during high speed writing activities, we now pre-fetch node IDs in batches, which eliminates an annoying performance bottleneck.
• For very large graphs, we have move the metadata about the page cache, into the page cache, which allows more of the actual graph to be contained within memory and delays the need for caching to disk.

Cluster operations are faster and more secure in Enterprise Edition

Graph Analytics with Graph Algorithms to Extend OLTP Functionality

• Community detection algorithms to evaluate how your graph is clustered or partitioned
• Pathfinding algorithms to quickly find the shortest path or evaluate route availability and quality
• Centrality algorithms like PageRank to determine the importance of distinct nodes in the network

Why Do Graph Analytics Matter?

• Developers build real-time graph traversal applications, which quickly become hungry to grow and need more data to connect and traverse.
• Big data IT can supply that new data and increase the utility of the data lake.
• Data scientists can use this existing and new data to explore, test and develop new algorithms and share them back with developers to operationalize in the application.
• The data scientists’ algorithms not only operate upon the expanded graph, they can add data and connections to the graph itself, essentially making it smarter for each subsequent query as the application operates.
• If you look at the interaction of these teams, you’ll notice a loop where developers, big data IT and data scientists help the graph application grow by itself. This is the foundation of artificial intelligence, and we call it the AI workflow loop enabled by the Graph Platform.

• Knowledge Graph workflow loops involve big data IT, data scientists and traditional business analysts who all want a common catalog (ontology) of data assets and metadata, and these Knowledge Graphs want to represent how all these assets are related.
• Digital Transformation workflow loops between big data IT, business analysts who know both data and business processes, and the C-level initiative holder – like a Chief Data Officer or Chief Security Officer, or Chief of Compliance – all of whom need to see and understand how their information assets and business processes relate to each other.

Cypher for Apache Spark

• Multiple named graphs which allow Cypher to identify specific graphs upon which to operate.
• Compositional graph queries allows users to save graph query results as graphs in addition to Cypher’s default of returning tables. Supporting this allows compositional queries, which can be chained together in a function chain of graph algorithms, and this provides the building blocks for multiple graph-handling queries.
• Path control: New language features are in gestation to give user control over path uniqueness while defining traversal types like “walks,” “trails” and “paths” as specific patterns.
• Regular path expressions apply regular expression syntax in concisely stating complex path patterns when extracting subgraphs. It applies a powerful path expression language based on the leading-edge research language GXPath.

Data Integration

Neo4j ETL

• Connect to popular databases via JDBC.
• Arrange table structures as labeled nodes and identify JOINs and JOIN tables as relationships.
• Map or change labels and properties prior to execution.
• Data is exported in graph-ready CSVs and fed to the Neo4j importer.
• Graph connections are materialized through relational JOINs as data is imported, and then they are persisted permanently after import.

Data Lake Integrator

• Build metadata catalogs: Discover definitions of data objects and their relationships with each other to weave a highly connected view of information in the data lake. The resulting metadata graph will help data exploration, impact analysis and compliance initiatives.
  
• Wrangle data: Combine data from the lake with other sources including Neo4j, wrangling it with Cypher – the SQL for graphs. Composable queries allow Cypher queries to return data in an in-memory graph format using Apache Spark™, as well as in tabular or CSV formats.
  
• Conduct graph analytics: Import data directly from the data lake into the Neo4j Graph Platform for faster and intuitive graph analytics using graph algorithms such as PageRank, community detection and path finding to discover new insights.
  
• Operationalize data: Import data directly from the data lake into the Neo4j Graph Platform for real-time transactional and traversal applications.
  
• Snapshot graphs: The output of the Data Lake Integrator is a properly structured CSV for reconstituting Neo4j graphs. These files can be saved as snapshots, versioned, diffed, reused and backed up in HDFS.
  

Discovery and Graph Visualization

The post Neo4j: From Graph Database to Graph Platform appeared first on Neo4j Graph Database.

What’s New in Neo4j 3.3 – All Editions:

Cypher and Write Performance

Data Import, Integration & ETL

RDBMS-to-graph and bulk importing improvements let you hit the ground running with Neo4j.

Upgrades for Graph Analytics

Use graph algorithms to reveal unseen patterns in your connected data.

What’s New in Neo4j 3.3 – Enterprise Edition Only Features:

Neo4j Desktop

The Neo4j Desktop package is your mission control for organizing and managing all of your Neo4j projects.

Neo4j Database Kernel Improvements

Scalability Improvements: Load Balancing, Security & More

• Intra-server encryption keeps all server-to-server transmissions safe, from data center to data center and across cloud zones – all managed in Bolt.
• The Bolt driver’s new least-connected load balancing replaces round-robin selection in order to maintain high cluster throughput under high demand conditions.
• Also, node ids are fetched in batch, removing an otherwise pesky bottleneck revealed when clusters are under heavy activity.
• Clusters now support IPv6 which sets the stage for trillions of device and IP connections.

Conclusion

The post Neo4j Graph Database 3.3 Release: Everything You Need to Know appeared first on Neo4j Graph Database.

Making Cypher More Accessible to Data Scientists

CAPS: A Closer Look

Conclusion

The post Cypher – the SQL for Graphs – Is Now Available for Apache Spark appeared first on Neo4j Graph Database.

GraphQL Summit

GRANDstack

Neo4j-GraphQL Apollo Launchpad Challenge

Fork our example Apollo Launchpad and win a neo4j-graphql hacker T-shirt and sticker!

The post Neo4j & GRANDstack at GraphQL Summit [+Developer Challenge] appeared first on Neo4j Graph Database.

Neo4j Powers Personalized Promotion & Product Recommendation Engines

The Challenges with Traditional Systems

Data stored in relational databases and other silos is too disconnected and slow for real-time recommendations.

With a graph database, a retailer’s entire inventory, supply chain, customer data and other systems are all connected.

Case Study: Walmart

Case Study: Top-Ten Retail Company

Conclusion

The post Retail & Neo4j: Personalized Promotion & Product Recommendations appeared first on Neo4j Graph Database.

Our Stack

• KeyLines – for the visualization front-end
• GraphQL – to fetch data from the GitHub API
• Neo4j – as a graph datastore ‘cache’ for our GitHub data
• Angular – to neatly tie the project together

Loading a GitHub Account

Loading Neo4j’s 20 most recently updated repos

1. Send an event to the service provider.
2. The service auto-generates some Cypher to query the Neo4j database:
   

   MATCH (User {login:"christian-cam"})-[PullRequest:PULL_REQUEST]->(Repository)
      	RETURN User, PullRequest, Repository
   

   
   
3. If the response is blank, it sends a GraphQL query to the GitHub API:
   

   query ($login: String!) {
     user(login: $login) {
   	id
   	name
   	avatarUrl
   	login
   	company
   	pullRequests(first: 50, states: [MERGED], orderBy: {field: CREATED_AT, direction: DESC}) {
     	nodes {
       	id
       	title
       	number
       	commits {
         	totalCount
       	}
       	repository {
         	id
         	name
        	 owner {
           	id
           	login
           	avatarUrl
         	}
       	}
     	}
   	}
     }
   },
   {‘login’: ‘christian-cam’}
   

   
   
4. The GitHub API returns some data, which is cached in our Neo4j instance. It’s then loaded into KeyLines and styled according to your customization code.

My graph data model. It looks complex, but KeyLines will help us explore it manageably

Understanding the Structure with Automated Layouts

Some of Neo4j’s cross-repo community heroes

• Grey = pull requests
• Red = issues raised
• Blue = pull request reviews

Filtering the Graph by Time

Pull requests (grey), issues (red) and reviews (blue) all see a spike in September 2017

A GitHub pull request spike in September

Social Network Analysis

Try It Yourself

The post Going Meta: Exploring the Neo4j Graph Database…as a Graph appeared first on Neo4j Graph Database.

Unprecedented Volumes of Highly Connected Data

Stay Tuned for More Coverage of the Paradise Papers

The post Neo4j: The Power behind the Paradise Papers appeared first on Neo4j Graph Database.

Our friends from the ICIJ (International Consortium of Investigative Journalists) just announced the Paradise Papers this past week, a new trove of leaked documents from the law firm Appleby and trust company Asiaciti.

Similar to the Panama Papers before (which we covered in-depth here), we’ve learned from those records that a large number of people and organizations use shell companies in tax havens and offshore jurisdictions to hide, move and spend huge amounts of money (billions to trillions of USD) without the necessary fiscal oversight.

In the last few days we saw a huge number of reports being published covering activities of companies like Nike, Apple, unsavory involvement by the Queen’s investment group, connections of Russian investments to politicians like Wilbur Ross or companies like Facebook and Twitter and many more.

The more than 13 million documents, emails and database records have been analysed using text analysis, full-text- and faceted-search and – most interesting to us – graph visualization and search.

Special thanks go to the the Pulitzer Center on Crisis Reporting for supporting visual elements of the project, Neo4j and Linkurious for database support.

We are especially proud that Manuel Villa, whose position was sponsored by our “Connected Data Fellowship” was able to contribute to the research and data work.

As before, the leaked information was added to the already large body of data from previous leaks in a comprehensive Neo4j database that was available both to the data team as well as the investigative journalists.

The ICIJ published a fraction of the Paradise Papers data as part of their Power Players visualization at the same time as the reported stories. There are about 1000 nodes and 3000 relationships in this preliminary dataset.

We’re expecting the ICIJ to release larger parts of the dataset soon, and we will keep you updated with further findings.

Data Model

The data model the ICIJ used for the Panama Papers is quite straightforward. We’ve applied this same data model to the available Paradise Papers dataset. The model includes:

• A company, trust or fund created in a low-tax, offshore jurisdiction by an agent (Entity)
• People or companies who play a role in an offshore entity (Officer)
• Addresses (Address)
• Law-firms or middlemen (Intermediary) that asks an offshore service provider to create an offshore firm for a client.

Each of these carries different properties, including name, address, country, status, start and end date, validity information and more.

Relationships between the elements capture the roles people or other companies play in the offshore entities (often shell companies), we see many officer_of relationships for directors, shareholders, beneficiaries, etc.

Other relationships capture similar addresses or the responsibility of creating a shell company by a law firm (intermediary_of).

Until the data is officially published we used the information from the ICIJ’s website for some examples on the reported stories.

Here is the full graph of (currently) independent reports that will come together in the full dataset:

This initial Neo4j graph database consists of information about 212 legal entities, 669 officers connected to those entities, which were established through 15 different intermediaries (often law firms used to incorporate the legal entities).

We can now use the data in Neo4j for interactive graph visualization, but also for full graph querying and later for the application graph algorithms (Part 2 of this blog series).

Basic statistics over the data tell us that we have these entities:

MATCH (n)
RETURN labels(n), count(*)
ORDER BY count(*) DESC

╒════════════════╤══════════╕
│"labels(n)"     │"count(*)"│
╞════════════════╪══════════╡
│["Officer"]     │675       │
├────────────────┼──────────┤
│["Entity"]      │212       │
├────────────────┼──────────┤
│["Address"]     │134       │
├────────────────┼──────────┤
│["Intermediary"]│15        │
├────────────────┼──────────┤
│["Other"]       │11        │
└────────────────┴──────────┘

The distribution of degrees for our entities shows a typical power law: there are some addresses which have 127 companies registered and some people who have almost 90 shell companies registered to their name.

Remember, this is a tiny fraction of the Paradise Papers data.

MATCH (n)
WITH labels(n) AS type, size( (n)--() ) AS degree
RETURN type,
       max(degree) AS max, round(avg(degree)) AS avg, round(stdev(degree)) AS stdev

╒════════════════╤═════╤═════╤═══════╕
│"type"          │"max"│"avg"│"stdev"│
╞════════════════╪═════╪═════╪═══════╡
│["Other"]       │14   │4    │4      │
├────────────────┼─────┼─────┼───────┤
│["Address"]     │127  │5    │15     │
├────────────────┼─────┼─────┼───────┤
│["Intermediary"]│44   │10   │14     │
├────────────────┼─────┼─────┼───────┤
│["Officer"]     │89   │4    │8      │
├────────────────┼─────┼─────┼───────┤
│["Entity"]      │112  │12   │13     │
└────────────────┴─────┴─────┴───────┘

These are our relationships, so you see it’s mostly officers and addresses for entities and officers.

MATCH ()-[r]->()
RETURN type(r), count(*)
ORDER BY count(*) DESC

╒════════════════════╤══════════╕
│"type(r)"           │"count(*)"│
╞════════════════════╪══════════╡
│"officer_of"        │2079      │
├────────────────────┼──────────┤
│"registered_address"│639       │
├────────────────────┼──────────┤
│"connected_to"      │86        │
├────────────────────┼──────────┤
│"intermediary_of"   │67        │
├────────────────────┼──────────┤
│"same_name_as"      │28        │
├────────────────────┼──────────┤
│"same_id_as"        │2         │
└────────────────────┴──────────┘

We can break down the officer_of relationship by link property, then we see this:

MATCH ()-[r:officer_of]->()
RETURN toLower(r.link), count(*)
ORDER BY count(*) DESC
LIMIT 10

╒═══════════════════════════╤══════════╕
│"toLower(r.link)"          │"count(*)"│
╞═══════════════════════════╪══════════╡
│"director"                 │661       │
├───────────────────────────┼──────────┤
│"is shareholder of"        │326       │
├───────────────────────────┼──────────┤
│"alternate director"       │238       │
├───────────────────────────┼──────────┤
│"secretary"                │209       │
├───────────────────────────┼──────────┤
│"appleby assigned attorney"│102       │
├───────────────────────────┼──────────┤
│"vice-president"           │65        │
├───────────────────────────┼──────────┤
│"auditor"                  │54        │
├───────────────────────────┼──────────┤
│"president"                │52        │
├───────────────────────────┼──────────┤
│"ultimate beneficial owner"│48        │
├───────────────────────────┼──────────┤
│"is signatory for"         │41        │
└───────────────────────────┴──────────┘

What is interesting here, is the number of “appleby assigned attorneys” and “ultimate beneficial owner” which are not visible in the fiscal records.

Jurisdictions

This version of the dataset contains limited address information, but we can check to see the distribution of addresses across countries:

MATCH (n:Address) WHERE exists(n.country)
RETURN n.country, count(*)
ORDER BY count(*) DESC
LIMIT 10

╒═══════════╤══════════╕
│"n.country"│"count(*)"│
╞═══════════╪══════════╡
│"US"       │23        │
├───────────┼──────────┤
│"BM"       │12        │
├───────────┼──────────┤
│"BR"       │7         │
├───────────┼──────────┤
│"GB"       │7         │
├───────────┼──────────┤
│"IM"       │6         │
├───────────┼──────────┤
│"KY"       │4         │
├───────────┼──────────┤
│"ID"       │4         │
├───────────┼──────────┤
│"JO"       │3         │
├───────────┼──────────┤
│"HK"       │3         │
├───────────┼──────────┤
│"BS"       │3         │
└───────────┴──────────┘

One important question we can ask is: What are the most popular offshore jurisdictions used by people in other countries?

For example, for people with addresses in the U.S., what are the most common offshore jurisdictions of Officer’s by country of address

// Most popular offshore jurisdictions for Officer's by country of address
MATCH (a:Address {country: "US"})--(o:Officer)--(e:Entity)
RETURN e.jurisdiction_description AS jurisdiction, COUNT(*) AS num
ORDER BY num DESC LIMIT 10

╒══════════════════════════╤═════╕
│"jurisdiction"            │"num"│
╞══════════════════════════╪═════╡
│"Bermuda"                 │94   │
├──────────────────────────┼─────┤
│"Cayman Islands"          │74   │
├──────────────────────────┼─────┤
│"United States of America"│3    │
├──────────────────────────┼─────┤
│"State of Delaware"       │1    │
└──────────────────────────┴─────┘

We can see that the most common jurisdictions of entities with connections to people with addresses in the U.S. are Bermuda and the Cayman Islands, which also confirms common knowledge and the reason why so few Americans were in the Panama Papers, since Panama is not a common offshore jurisdiction for Americans.

Of course keep in mind that this is only querying a subset of the data. In our next blog post we’ll look at analyzing the full dataset.

Wilbur Ross

The current U.S. Secretary of Commerce, Wilbur Ross, was revealed to have connections to offshore companies, as reported by the ICIJ earlier last week.

// What are the jurisdictions of Ross's connected entities?
MATCH (o:Officer)--(e:Entity)
WHERE o.name CONTAINS "Ross"
RETURN e.jurisdiction_description AS jurisdiction, COUNT(*) AS num
ORDER BY num DESC

╒═══════════════════╤═════╕
│"jurisdiction"     │"num"│
╞═══════════════════╪═════╡
│"Cayman Islands"   │16   │
├───────────────────┼─────┤
│"State of Delaware"│1    │
└───────────────────┴─────┘

We see that Ross is connected to 17 legal entities, 16 of which are registered in the Cayman Islands:

// Wilbur Ross’s connections in the Paradise Papers
MATCH (o:Officer)-->(e:Entity)-[:intermediary_of]-(i:Intermediary)
WHERE o.name CONTAINS "Ross"
MATCH (e)--(o2:Officer)
RETURN *

We could write a similar Cypher statement to query for Ross’s second-degree network, without specifying node labels or relationship types:

MATCH p=(o:Officer)-[*..2]-()
WHERE o.name CONTAINS "Ross"
RETURN p

Queen Elizabeth Investment House

The Duchy of Lancaster – Queen Elizabeth II’s private estate and portfolio – appears in the Paradise Papers dataset. An investment of several million dollars was made in a Cayman entity, Dover Street VI Cayman Fund by the Duchy. While the Queen’s estate regularly reports some domestic investments and holdings, offshore holdings, including the Dover Street VI Cayman Fund investment had not been previously reported.

We can write a Cypher query for all two-degree connections to the Duchy:

MATCH p=(o:Officer {name: "The Duchy of Lancaster"})-[*..2]-()
RETURN p

We can see a common pattern for the orchestration of offshore entities. If we expand two of the Officer nodes, “Robinson – Lynniece” and “Sutherland – Linda M” we see that they are serve as officers for many other offshore entities:

These individuals serve as corporate services managers, where they may serve as officers for hundreds of offshore entities.

Looking Ahead

This post was an introduction to the Paradise Papers dataset, with some examples from the recently published investigations.

In our next post, we will explore how we can apply graph algorithms, virtual graph projections, more advanced querying and spatial analysis to the entire Paradise Papers dataset.

References

• About the Paradise Papers – ICIJ
• The Offshore Leaks Online Search
• Presentation by Mar Cabra on the Panama Papers investigation
• Neo4j Sandbox for the Panama Papers dataset
• Blog Post: Analyzing the Panama Papers with Neo4j

The post Analyzing the Paradise Papers with Neo4j: A Closer Look at Queries, Data Models & More appeared first on Neo4j Graph Database.

Graph databases are the best way to explore the relationships between these people and entities — it’s much more intuitive to use for this purpose than a SQL database or other types of NoSQL databases. In addition, graphs allow to understand these networks in a very intuitive way and easily discover connections.

Graph Querying

The Data Model

Exploratory Queries

MATCH (n) RETURN labels(n) AS labels, COUNT(*) AS count ORDER BY count DESC
╒════════════════╤═══════╕
│"labels"        │"count"│
╞════════════════╪═══════╡
│["Officer"]     │77012  │
├────────────────┼───────┤
│["Address"]     │59228  │
├────────────────┼───────┤
│["Entity"]      │24957  │
├────────────────┼───────┤
│["Intermediary"]│2031   │
├────────────────┼───────┤
│["Other"]       │186    │
└────────────────┴───────┘

MATCH ()-[r]->() RETURN type(r), COUNT(*) ORDER BY COUNT(*) DESC
╒════════════════════╤══════════╕
│"type(r)"           │"COUNT(*)"│
╞════════════════════╪══════════╡
│"OFFICER_OF"        │221112    │
├────────────────────┼──────────┤
│"REGISTERED_ADDRESS"│128311    │
├────────────────────┼──────────┤
│"CONNECTED_TO"      │10552     │
├────────────────────┼──────────┤
│"INTERMEDIARY_OF"   │4063      │
├────────────────────┼──────────┤
│"SAME_NAME_AS"      │416       │
├────────────────────┼──────────┤
│"SAME_ID_AS"        │2         │
└────────────────────┴──────────┘

MATCH (n) WITH labels(n) AS type, SIZE( (n)--() ) AS degree
RETURN type, MAX(degree) AS max, ROUND(AVG(degree)) AS avg, ROUND(STDEV(degree)) AS stdev
╒════════════════╤═════╤═════╤═══════╕
│"type"          │"max"│"avg"│"stdev"│
╞════════════════╪═════╪═════╪═══════╡
│["Other"]       │2891 │44   │236    │
├────────────────┼─────┼─────┼───────┤
│["Address"]     │9268 │2    │59     │
├────────────────┼─────┼─────┼───────┤
│["Intermediary"]│115  │5    │8      │
├────────────────┼─────┼─────┼───────┤
│["Officer"]     │2726 │4    │20     │
├────────────────┼─────┼─────┼───────┤
│["Entity"]      │312  │11   │13     │
└────────────────┴─────┴─────┴───────┘

The Shortest Path from the Queen of England to Rex Tillerson

MATCH p=shortestPath((rex:Officer)-[*]-(queen:Officer))
WHERE rex.name = "Tillerson - Rex" AND queen.name = "The Duchy of Lancaster"
RETURN p

MATCH p=allShortestPaths((rex:Officer)-[*]-(queen:Officer))
WHERE rex.name = "Tillerson - Rex" AND queen.name = "The Duchy of Lancaster"
RETURN p

Graph Algorithms

CALL algo.pageRank(null,null,{write:true,writeProperty:'pagerank_g'})

MATCH (e:Entity) WHERE exists(e.pagerank_g)
RETURN e.name AS entity, e.jurisdiction_description AS jurisdiction,
       e.pagerank_g AS pagerank ORDER BY pagerank DESC LIMIT 15

╒═════════════════════════════════════════════════╤════════════════╤══════════════════╕
│"entity"                                         │"jurisdiction"  │"pagerank"        │
╞═════════════════════════════════════════════════╪════════════════╪══════════════════╡
│"WORLDCARE LIMITED"                              │"Bermuda"       │18.110508499999998│
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Ferrous Resources Limited"                      │"Isle of Man"   │17.326935999999996│
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"American Contractors Insurance Group Ltd."      │"Bermuda"       │15.6201275        │
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Gulf Keystone Petroleum Limited"                │"Bermuda"       │12.81925          │
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Warburg Pincus (Bermuda) Private Equity X, L.P."│"Bermuda"       │12.312412         │
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Madagascar Oil Limited"                         │"Bermuda"       │11.611646499999999│
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Coller International Partners IV-D, L.P."       │"Cayman Islands"│11.394854         │
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Milestone Insurance Co., Ltd."                  │"Bermuda"       │11.224089         │
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"CL Acquisition Holdings Limited"                │"Cayman Islands"│11.0752455        │
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Alpha and Omega Semiconductor Limited"          │"Bermuda"       │10.965910000000001│
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤
│"Coller International Partners V-A, L.P."        │"Cayman Islands"│10.8205005        │
├─────────────────────────────────────────────────┼────────────────┼──────────────────┤

Geo Analysis

Heatmap of Paradise Papers geocoded addresses. Try it live.

Heatmap of Panama Papers geocoded addresses

Annotated map of geocoded addresses in Paradise Papers showing registered address of Officer nodes and connected legal entities and jurisdictions. Try it live.

Entity Jurisdictions

MATCH (e:Entity)
WITH e.jurisdiction_description AS juris, COUNT(*) AS count
WHERE count > 20
RETURN *
ORDER BY count ASC

MATCH (a:Address)--(o:Officer)--(e:Entity)
WITH a.countries AS officer_country, e.jurisdiction_description AS juris,
COUNT(*) AS num
RETURN * ORDER BY num DESC

What Can You Find?

You can find the code for generating all visualizations in this post on GitHub.

Editor’s note: ICIJ has published this data with the following note: “There are legitimate uses for offshore companies and trusts. We do not intend to suggest or imply that any people, companies or other entities included in the ICIJ Offshore Leaks Database have broken the law or otherwise acted improperly. Many people and entities have the same or similar names. We suggest you confirm the identities of any individuals or entities located in the database based on addresses or other identifiable information.”

The post An In-Depth Graph Analysis of the Paradise Papers appeared first on Neo4j Graph Database.

The Growing Graph Database Space

The Game Is Only Beginning

The Neo4j Graph Platform

Coalescing around a Query Language

Cypher for Apache Spark

The Graph Community Is Growing

The post Welcome to the Graph Community, Amazon Neptune! appeared first on Neo4j Graph Database.

Using the Paradise Papers Explorer

Screencast: Neo4j Desktop for ICIJ

Thanks to the excellent work of the ICIJ team – including the Neo4j Connected Data Fellow Manuel Villa – for their great story highlighting the capability of graph technology with this release. The ICIJ data engineer Miguel Fiandor Gutiérrez thankfully led the effort to make the data available in its raw format and as the Neo4j Desktop package.

If you’re a data journalist, apply to our Data Journalism Accelerator Program if you think graphs can help turn your data into knowledge graphs.

The post The ICIJ Releases Neo4j Desktop Download of Paradise Papers appeared first on Neo4j Graph Database.

Introduction

Async API

Driver Initialization

import org.neo4j.driver.v1.AuthTokens;
import org.neo4j.driver.v1.Driver;
import org.neo4j.driver.v1.GraphDatabase;

Driver driver = GraphDatabase.driver("bolt://localhost",
                                     AuthTokens.basic("neo4j", "test"));

Sessions

import org.neo4j.driver.v1.Session;

Session session = driver.session();

import org.neo4j.driver.v1.StatementResultCursor;

CompletionStage<StatementResultCursor> cursorStage =
      session.runAsync("UNWIND range(1, 10) AS x RETURN x");

cursorStage.thenCompose(StatementResultCursor::listAsync)
                    .whenComplete((records, error) -> {
                           if (records != null) System.out.println( records );
                           else error.printStackTrace();
                           session.closeAsync();
                    });

import org.neo4j.driver.v1.StatementResultCursor;

CompletionStage<StatementResultCursor> cursorStage = session.runAsync("MATCH (n) RETURN n LIMIT 1");

cursorStage.thenCompose(StatementResultCursor::singleAsync)
                   .thenApply(record -> record.get( 0 ).asNode())
                   .thenApply(Node::labels)
                   .exceptionally(error -> {
                       error.printStackTrace();
                       return emptyList();
                   })
                   .thenAccept(labels -> System.out.println(labels))
                   .thenCompose(ignore -> session.closeAsync());

• CompletionStage<Record> nextAsync() – returns stage completed with next records in the result stream or null when end of stream has been reached. Stage can also be completed exceptionally when query fails.
• CompletionStage<ResultSummary> forEachAsync(Consumer<Record> action) – returns stage completed with summary and applies supplied action to every record of the result stream.

import rx.Observable;
import rx.subjects.PublishSubject;

Observable<Record> fetchRecords(Session session, String query) {
    PublishSubject<Record> subject = PublishSubject.create();
    session.runAsync(query)
                .thenCompose(cursor -> cursor.forEachAsync(subject::onNext))
                .whenComplete((summary, error) -> {
                    if (error != null) {
                        subject.onError( error );
                    } else {
                        System.out.println( summary );
                        subject.onCompleted();
                    }
                });
    return subject;
}

Observable<Record> recordsObservable = fetchRecords(session, "MATCH (n:Person) RETURN n");
recordsObservable.subscribe(
    record -> System.out.println(record),
    error -> error.printStackTrace(),
    () -> System.out.println("Query completed")
);

Transactions

• Session#readTransactionAsync(TransactionWork<CompletionStage<T>>)
• Session#writeTransactionAsync(TransactionWork<CompletionStage<T>>)

session.writeTransactionAsync(tx ->
  tx.runAsync("CREATE (n:Person) RETURN n")
     .thenCompose(StatementResultCursor::singleAsync)
).whenComplete((record, error) -> {
    if (error != null) error.printStackTrace();
    else System.out.println(record);
    session.closeAsync();
});

session.readTransactionAsync(tx ->
    tx.runAsync("MATCH (n:Person) RETURN n")
      .thenCompose(cursor -> cursor.forEachAsync(System.out::println))
).whenComplete((ignore, error) -> {
    if ( error != null ) error.printStackTrace();
    session.closeAsync();
});

Driver Termination

driver.close();

Use with Maven

<dependency>
    <groupId>org.neo4j.driver</groupId>
    <artifactId>neo4j-java-driver</artifactId>
    <version>1.5.0-beta03</version>
</dependency>

Other Notable Changes

• A new load-balancing strategy for Causal Clustering uses a least-connected strategy instead of round-robin, which might result in better performance and less degradation when some cluster members perform poorly due to network or other similar issues.
• Improved connection pooling: The Java driver now allows setting a limit on the amount of connections in the pool per server address via Config.build().withMaxConnectionPoolSize(25) and connection acquisition timeout via Config.build().withConnectionAcquisitionTimeout(10, TimeUnit.SECONDS)
• Maximum connection lifetime: The Java driver allows for the limiting lifetime of a connection, which can be configured using Config.build().withMaxConnectionLifetime(1, TimeUnit.HOURS)

Conclusion

The post Beta Release: Java Driver with Async API for Neo4j appeared first on Neo4j Graph Database Platform.

The GraphTour Basics

GraphTour Presenters

GraphClinics & Neo4j Solutions

The GraphTour Itinerary (So Far)

• Tel Aviv: Tuesday, 13 February 2018
• Madrid: Thursday, 15 February 2018
• Berlin: Tuesday, 27 February 2018
• London: Thursday, 1 March 2018
• Paris: Tuesday, 6 March 2018
• Stockholm: Thursday, 8 March 2018
• Amsterdam: Wednesday, 21 March 2018
• Milan: Wednesday, 11 April 2018
• United States: Locations and dates to be announced soon

Special Thanks to Our GraphTour Sponsors

See You on the Neo4j GraphTour!

The post Announcing the Neo4j GraphTour in 8+ Cities Worldwide appeared first on Neo4j Graph Database Platform.

Connected Data Is Creating New Business Opportunities

It’s all about connected data! Connecting data helps companies answer complex questions, such as “Is David’s credit card purchase a fraud, given his buying patterns, the type of product that he is buying, the time and location of the purchase, and his likes and dislikes?” or “From the thousands of products, what is Jane likely to buy next given her buying behavior, products she has reviewed, her purchasing power, and other influencing factors?”

Developers could write Java, Python, or even SQL code to get answers to such complex questions, but that would take hours or days to program and in some cases might be impractical. What if business users want answers to such ad hoc questions quickly, with no time for custom code or with no access to the technical expertise needed to write those programs?

While organizations have been leveraging connections in data for decades, the need for rapid answers amid radical changes in data volume, diversity, and distribution has driven enterprise architects to look for new approaches.

You Don’t Have to Take Our Word for It

The post Forrester Research: Graph Databases Vendor Landscape [Free Report] appeared first on Neo4j Graph Database Platform.

Summary

• Neo4j Graph Database version 3.2.x, either Community or Enterprise Edition, Linux or Windows (I use Windows 10)
• Internet browser to access the Neo4j Browser (I use Chrome)
• A graph database. Here I imported data from the Stack Overflow Questions dataset, which contains more than 31 million nodes, 77 million relationships and 260 million properties. The total database size on Windows 10 is about 20 GB.

1. Database Schema Analysis

1.1 Show the graph data model (the meta model)

// Show what is related, and how (the meta graph model)
CALL db.schema()

• User
• Post
• Tag

• User POSTED Post
• Post HAS_TAG Tag
• Post is PARENT_OF Post
• Post ANSWER another Post

1.2 Show existing constraints and indexes

// Display constraints and indexes
:schema

   ON :Post(answers) ONLINE
   ON :Post(createdAt) ONLINE
   ON :Post(favorites) ONLINE
   ON :Post(score) ONLINE
… …

   ON ( post:Post ) ASSERT post.postId IS UNIQUE
   ON ( tag:Tag ) ASSERT tag.tagId IS UNIQUE
   ON ( user:User ) ASSERT user.userId IS UNIQUE

1.3 Show all relationship types

// List relationship types
CALL db.relationshipTypes()

1.4 Show all node labels / types

// List node labels
CALL db.labels()

1.5 Count all nodes

// Count all nodes
MATCH (n) RETURN count(n)

1.6 Count all relationships

// Count all relationships
MATCH ()-[r]->() RETURN count(*)

1.7 Show data storage sizes

// Data storage sizes
:sysinfo

1.8 Sample data

// What kind of nodes exist
// Sample some nodes, reporting on property and relationship counts per node.
MATCH (n) WHERE rand() <= 0.1
RETURN
DISTINCT labels(n),
count(*) AS SampleSize,
avg(size(keys(n))) as Avg_PropertyCount,
min(size(keys(n))) as Min_PropertyCount,
max(size(keys(n))) as Max_PropertyCount,
avg(size( (n)-[]-() ) ) as Avg_RelationshipCount,
min(size( (n)-[]-() ) ) as Min_RelationshipCount,
max(size( (n)-[]-() ) ) as Max_RelationshipCount

2. Node Analysis

2.1 Count nodes by their labels / types

// List all node types and counts
MATCH (n) RETURN labels(n) AS NodeType, count(n) AS NumberOfNodes;

2.2 Property Analysis

2.2.1 List all properties of a node

// List all properties of a node
MATCH (u:User) RETURN keys(u) LIMIT 1

2.2.2 List all properties of a relationship

// List all properties of a relationship
MATCH ()-[t:POSTED]-() RETURN keys(t) LIMIT 1

2.2.3 Uniqueness of the property

// Calculate uniqueness of a property
MATCH (u:User)
RETURN count(DISTINCT u.name) AS DistinctName,
       count(u.name) AS TotalUser,
       100*count(DISTINCT u.name)/count(u.name) AS Uniqueness;

2.2.4 Nullability of the property

// Calculate nullability of a property
MATCH (u:User) WHERE u.name IS null RETURN count(u);

2.2.5 Min, Max, Average and Standard Deviation of the Values of a property

// Calculate min, max, average and standard deviation of the values of a property
MATCH (p:Post)
RETURN min(p.favorites) AS MinFavorites,
       max(p.favorites) AS MaxFavorites,
       avg(p.favorites) AS AvgFavorites,
       stDev(p.favorites) AS StdFavorites;

2.2.6 Occurrences of values of a property

// Find out most often used values for a property
MATCH (p:Post)
RETURN p.answers AS Answers, count(p.answers) AS CountOfAnswers
ORDER BY Answers ASC;

2.3 Node Rank (Centrality)

2.3.1 Importance of a user

// Calculate node rank / Centrality of a node
// i.e., the relevance of a node by counting the edges from other nodes:
// in-degree, out-degree and total degree.

MATCH (u:User)
WITH u,size( (u)-[:POSTED]->()) AS OutDepth, size( (u)<-[:POSTED]-()) AS InDepth
ORDER BY OutDepth, InDepth
WHERE u.name STARTS WITH 'T'
RETURN u.name, min(OutDepth), max(OutDepth), min(InDepth), max(InDepth)

… … … …

2.3.2 Importance of a post

2.4 Orphan Nodes

// orphans: node has no relationship
match (u:User)
with u,size( (u)-[:POSTED]->()) as posts where posts = 0
return u.name, posts;

… … … …

3. Relationship Analysis

3.1 Statistics on relationships

// Count relationships by type
match (u)-[p]-() with type(p) as RelationshipName,
count(p) as RelationshipNumber
return RelationshipName, RelationshipNumber;

MATCH ()-[r]->() RETURN type(r), count(*)

3.2 Find the shortest path between two nodes

// Find all shortest path between 2 nodes
MATCH path =
      allShortestPaths((u:User {name:"Darin Dimitrov"})-[*]-(me:User {name:"Michael Hunger"}))
RETURN path;

The shortest path between the two chosen users is 6.

3.3 Triangle detection

// Triangle detection:
match (u:User)-[p1:POSTED]-(x1),(u)-[p2:POSTED]-(x2),(x1)-[r1]-(x2)
where x1 <> x2 return u,p1,p2,x1,x2 limit 10;

// Count all triangles in the graph
match (u:User)-[p1:POSTED]-(x1),(u)-[p2:POSTED]-(x2),(x1)-[r1]-(x2)
where x1 <> x2 return count(p1);

A triadic closure is a common property of social graphs, where we observe that if two nodes are connected via a path involving a third node, there is an increased likelihood that the two nodes will become directly connected at some point in the future.

4. Using the APOC Library

1. Stop Neo4j service
2. Download and copy the most recent version of the APOC JAR file to the plugins folder under the database, e.g., graph.db\plugins
3. Add the following line to the neo4j.conf file: dbms.security.procedures.unrestricted=apoc.*
4. Start Neo4j service again

•  CALL apoc.meta.data()

•  CALL apoc.meta.graph()

•  CALL apoc.meta.stats()

{
  "labelCount": 1,
  "relTypeCount": 1,
  "propertyKeyCount": 3,
  "nodeCount": 107,
  "relCount": 352,
  "labels": {
    "Character": 107
  },
  "relTypes": {
    "(:Character)-[:INTERACTS]->()": 352,
    "()-[:INTERACTS]->(:Character)": 352,
    "()-[:INTERACTS]->()": 352
  }
}

•  CALL apoc.meta.schema()

•  CALL apoc.meta.subGraph({labels:['Character'],rels:['INTERECTS']})

CALL apoc.meta.subGraph({labels:[labels],rels:[rel-types],excludes:[label,rel-type,…​]})

5. Further Discussion

The post Data Profiling: A Holistic View of Data using Neo4j appeared first on Neo4j Graph Database Platform.

Why Customer Experience Personalization Is Critical

The Essential Role of Graph Technology

Case Study: Global Sporting Goods Retailer

Conclusion

The post Retail & Neo4j: Customer Experience Personalization appeared first on Neo4j Graph Database Platform.

[As community content, this post reflects the views and opinions of the particular author and does not necessarily reflect the official stance of Neo4j.]

This guide runs through the basic steps for importing the bitcoin blockchain into a Neo4j graph database.

The whole process is just about taking data from one format (blockchain data), and converting it into another format (a graph database). The only thing that makes this slightly trickier than typical data conversion is that it’s helpful to understand of the structure of bitcoin data before you get started.

However, once you have imported the blockchain into Neo4j, you can perform analysis on the graph database that would not be possible with SQL databases. For example, you can follow the path of bitcoins to see if two different addresses are connected:

Screenshot of connected Bitcoin Addresses in the Neo4j Browser.

In this guide I will cover:

1. How bitcoin works, and what the blockchain is.
2. What blockchain data looks like.
3. How to import the blockchain data into Neo4j.

This isn’t a complete tutorial on how to write your own importer tool. However, if you’re interested, you can find my bitcoin-to-neo4j code on GitHub, although I’m sure you could write something cleaner after reading this guide.

1. What Is Bitcoin?

Bitcoin is a computer program.

It’s a bit like uTorrent; you run the program, it connects to other computers running the same program, and it shares a file. However, the cool thing about bitcoin is that anyone can add data to this shared file, and any data already written to the file cannot be tampered with.

As a result, Bitcoin creates a secure file that is shared on a distributed network.

What can you do with this?

In bitcoin, each piece of data that gets added to this file is a transaction. Therefore, this decentralised file is being used as a “ledger” for a digital currency (i.e., cryptocurrency).

This ledger is called the blockchain.

Where can I find the blockchain?

If you run the Bitcoin Core program, the blockchain will be stored in a folder on your computer:

• Linux: ~/.bitcoin/blocks
• Windows: ~/Library/Application Support/Bitcoin/blocks
• Mac: C:\Users\YourUserName\Appdata\Roaming\Bitcoin\blocks

When you open this directory you should notice that instead of one big file, you will find multiple files with the name blkXXXXX.dat. This is the blockchain data, but split across multiple smaller files.

2. What Does the Blockchain Look Like?

The blk.dat files contain serialized data of blocks and transactions.

Blocks

Blocks are separated by magic bytes, which are then followed by the size of the upcoming block.

Each block then begins with a block header:

A block is basically a container for a list of transactions. The header is like the metadata at the top.

000000206c77f112319ae21489b66774e8acd379044d4a23ea7498000000000000000000821fe1890186779b2cc232d5dbecfb9119fd46f8a9cfd1141649ff1cd907374487d8ae59e93c011832ec0399

Transactions

After the block header, there is a byte that tells you the upcoming number of transactions in the block. After that, you get serialized transaction data, one after the other.

A transaction is just another piece of code again, but they are more structurally interesting.

Each transaction has the same pattern:

1. Select Outputs (we call these Inputs).

• Unlock these inputs so that they can be spent.

• Lock these outputs to a new address.

So after a series of transactions, you have a transaction structure that looks like something this:

This is a simplified diagram of what the blockchain looks like. As you can see, it looks like a graph.

0200000001f2f7ee9dda0ba82031858d30d50d3205eea07246c874a0488532014d3b653f03000000006a47304402204df1839028a05b5b303f5c85a66affb7f6010897d317ac9e88dba113bb5a0fe9022053830b50204af15c85c9af2b446338d049672ecfdeb32d5124e0c3c2256248b7012102c06aec784f797fb400001c60aede8e110b1bbd9f8503f0626ef3a7e0ffbec93bfeffffff0200e1f505000000001976a9144120275dbeaeb40920fc71cd8e849c563de1610988ac9f166418000000001976a91493fa3301df8b0a268c7d2c3cc4668ea86fddf81588ac61610700

3. How to Import the Blockchain into Neo4j

Well, now we know what the blockchain data represents (and that it looks a lot like a graph), we can go ahead and import it into Neo4j. We do this by:

1. Reading through the blk.dat files.
2. Decoding each block and transaction we run into.
3. Converting the decoded block/transaction into a Cypher query.

Here’s a visual guide to how I represent Blocks, Transactions and Addresses in the database:

Blocks

1. CREATE a :block node, and connect it to the previous block it builds upon.
   • SET each field from the block header as properties on this node.
2. CREATE a :coinbase node coming off each block, as this represents the “new” bitcoins being made available by the block.
   • SET a value property on this node, which is equal to the block reward for this block.

Transactions

1. CREATE a :tx node, and connect it to the :block we had just created.
   • SET properties (version, locktime) on this node.
2. MERGE existing :output nodes and relate them [:in] to the :tx.
   • SET the unlocking code as a property on the relationship.
3. CREATE new :output nodes that this transaction creates.
   • SET the respective values and locking codes on these nodes.

If the locking code on an :output contains an address…

1. CREATE an :address node, and connect the output node to it.
   • SET the address as a property on this node.
   • Note: If different outputs are connected to the same address, then they will be connected to the same address node.

4. Cypher Queries

Here are some example Cypher queries you could use for the basis of inserting blocks and transactions into Neo4j.

Note: You will need to decode the block headers and transaction data to get the parameters for the Cypher queries.

Block

MERGE (block:block {hash:$blockhash})
CREATE UNIQUE (block)-[:coinbase]->(:output:coinbase)
SET
   block.size=$size,
   block.prevblock=$prevblock,
   block.merkleroot=$merkleroot,
   block.time=$timestamp,
   block.bits=$bits,
   block.nonce=$nonce,
   block.txcount=$txcount,
   block.version=$version,

MERGE (prevblock:block {hash:$prevblock})
MERGE (block)-[:chain]->(prevblock)

{
	"blockhash": "00000000000003e690288380c9b27443b86e5a5ff0f8ed2473efbfdacb3014f3",
	"version": 536870912,
	"prevblock": "000000000000050bc5c1283dceaff83c44d3853c44e004198c59ce153947cbf4",
	"merkleroot": "64027d8945666017abaf9c1b7dc61c46df63926584bed7efd6ed11a6889b0bac",
	"timestamp": 1500514748,
	"bits": "1a0707c7",
	"nonce": 2919911776,
	"size": 748959,
	"txcount": 1926,
}

Transaction

MATCH (block :block {hash:$hash})
MERGE (tx:tx {txid:$txid})
MERGE (tx)-[:inc {i:$i}]->(block)
SET tx += {tx}

WITH tx
FOREACH (input in $inputs |
         MERGE (in :output {index: input.index})
         MERGE (in)-[:in {vin: input.vin, scriptSig: input.scriptSig, sequence: input.sequence, witness: input.witness}]->(tx)
         )

FOREACH (output in $outputs |
         MERGE (out :output {index: output.index})
         MERGE (tx)-[:out {vout: output.vout}]->(out)
         SET
             out.value= output.value,
             out.scriptPubKey= output.scriptPubKey,
             out.addresses= output.addresses
         FOREACH(ignoreMe IN CASE WHEN output.addresses <> '' THEN [1] ELSE [] END |
                 MERGE (address :address {address: output.addresses})
                 MERGE (out)-[:locked]->(address)
                 )
        )

Note: This query uses the FOREACH hack, which acts as a conditional and will only create the :address nodes if the $addresses parameter actually contains an address (i.e., if it is not empty).

{
   "txid":"2e2c43d9ef2a07f22e77ed30265cc8c3d669b93b7cab7fe462e84c9f40c7fc5c",
   "hash":"00000000000003e690288380c9b27443b86e5a5ff0f8ed2473efbfdacb3014f3",
   "i":1,
   "tx":{
      "version":1,
      "locktime":0,
      "size":237,
      "weight":840,
      "segwit":"0001"
   },
   "inputs":[
      {
         "vin":0,
         "index":"0000000000000000000000000000000000000000000000000000000000000000:4294967295",
         "scriptSig":"03779c110004bc097059043fa863360c59306259db5b0100000000000a636b706f6f6c212f6d696e65642062792077656564636f646572206d6f6c69206b656b636f696e2f",
         "sequence":4294967295,
         "witness":"01200000000000000000000000000000000000000000000000000000000000000000"
      }
   ],
   "outputs":[
      {
         "vout":0,
         "index":"2e2c43d9ef2a07f22e77ed30265cc8c3d669b93b7cab7fe462e84c9f40c7fc5c:0",
         "value":166396426,
         "scriptPubKey":"76a91427f60a3b92e8a92149b18210457cc6bdc14057be88ac",
         "addresses":"14eJ6e2GC4MnQjgutGbJeyGQF195P8GHXY"
      },
      {
         "vout":1,
         "index":"2e2c43d9ef2a07f22e77ed30265cc8c3d669b93b7cab7fe462e84c9f40c7fc5c:1",
         "value":0,
         "scriptPubKey":"6a24aa21a9ed98c67ed590e849bccba142a0f1bf5832bc5c094e197827b02211291e135a0c0e",
         "addresses":""
      }
   ]
}

5. Results

If you have inserted the blocks and transactions using the Cypher queries above, then these are some examples the kind of results you can get out of the graph database.

Block

MATCH (block :block)<-[:inc]-(tx :tx)
WHERE block.hash='$blockhash'
RETURN block, tx

Transaction

MATCH (inputs)-[:in]->(tx:tx)-[:out]->(outputs)
WHERE tx.txid='$txid'
OPTIONAL MATCH (inputs)-[:locked]->(inputsaddresses)
OPTIONAL MATCH (outputs)-[:locked]->(outputsaddresses)
OPTIONAL MATCH (tx)-[:inc]->(block)
RETURN inputs, tx, outputs, block, inputsaddresses, outputsaddresses

Address

MATCH (address :address {address:'1PNXRAA3dYTzVRLwWG1j3ip9JKtmzvBjdY'})<-[:locked]-(output :output)
WHERE address.address='$address'
RETURN address, output

Paths

Finding paths between transactions and addresses is probably the most interesting thing you can do with a graph database of the bitcoin blockchain, so here are some examples of Cypher queries for that:

MATCH (start :output {index:'$txid:vout'}), (end :output {index:'$txid:out'})
MATCH path=shortestPath( (start)-[:in|:out*]-(end) )
RETURN path

MATCH (start :address {address:'$address1'}), (end :address {address:'$address2'})
MATCH path=shortestPath( (start)-[:in|:out|:locked*]-(end) )
RETURN path

Conclusion

This has been a simple guide on how you can take the blocks and transactions from blk.dat files (the blockchain) and import them into a Neo4j database.

I think it’s worth the effort if you’re looking to do serious graph analysis on the blockchain. A graph database is a natural fit for bitcoin data, whereas using an SQL database for bitcoin transactions feels like trying to shove a square peg into a round hole.

I’ve tried to keep this guide compact, so I haven’t covered things like:

1. Reading through the blockchain. Reading the blk.dat files is easy enough. However, the annoying thing about these files is that the blocks are not written to these files in sequential order, which makes setting the height on a block or calculating the fee for a transaction a bit trickier (but you can code around it).
2. Decoding blocks and transactions. If you want to use the Cypher queries above, you will need to get the parameters you require by decoding the block headers and raw transaction data as you go. You could write your own decoders, or you could try using an existing bitcoin library.
3. Segregated Witness. I’ve only given a Cypher query for an “original” style transaction, which was the only transaction structure used up until block 481,824. However, the structure of a segwit transaction is only slightly different (but it might need its own Cypher query).

Nonetheless, hopefully this guide has been somewhat helpful.

But as always, if you understand how the data works, converting it to a different format is just a matter of sitting down and writing the tool.

Good luck.

The post How to Import the Bitcoin Blockchain into Neo4j [Community Post] appeared first on Neo4j Graph Database Platform.

Meltdown & Spectre: Frequently Asked Questions

Further Updates Are Forthcoming

Resources on Meltdown & Spectre

• https://meltdownattack.com/
• Side-Channel Analysis Facts and Intel Products
• AMD Processors: Google Project Zero, Spectre and Meltdown

The post What You Need to Know about How Meltdown and Spectre Affect Neo4j appeared first on Neo4j Graph Database Platform.

How Neo4j Transforms Ecommerce Delivery Service Routing

Case Study: eBay

“Our Neo4j solution is literally thousands of times faster than the prior MySQL solution, with queries that require 10-100 times less code. At the same time, Neo4j allowed us to add functionality that was previously not possible,” said Volker Pacher, Senior Developer for eBay.

Conclusion

The post Retail & Neo4j: Ecommerce Delivery Service Routing appeared first on Neo4j Graph Database Platform.

How Neo4j Enables Crystal-Clear Supply Chain Visibility

Case Study: Transparency-One

Conclusion

The post Retail & Neo4j: Supply Chain Visibility & Management appeared first on Neo4j Graph Database Platform.