Creating, Writing and Reading Jena TDB2 Datasets

Posted on April 10, 2018 by Henriette Harmse

Jena TDB2 can be used as an RDF datastore. Note that TDB (version 1 of Jena TDB) and TDB2 are not compatible with each other. TDB2 is per definition transactional (while TDB is not). In this post I give a simple example that

create a new Jena TDB2 dataset,
create a write transaction and write data to the datastore,
create a read transaction and read the data from the datastore, and
release resources associated with the dataset on writing and reading is done.

Create TDB2 Dataset

To create a Jena TDB2 dataset, we use the TDB2Factory. Note that the class name is TDB2Factory and not TDBFactory. We need to specify a directory where our dataset will be created. Multiple datasets cannot be written to the same directory.

Path path = Paths.get(".").toAbsolutePath().normalize();      
String dbDir = path.toFile().getAbsolutePath() + "/db/"; 
Location location = Location.create(dbDir);      
Dataset dataset = TDB2Factory.connectDataset(location);

Create WRITE Transaction and Write

dataset.begin(ReadWrite.WRITE);
UpdateRequest updateRequest = UpdateFactory.create(
  "INSERT DATA { " 
  + " \"Grace Hopper\" .}");
UpdateProcessor updateProcessor = 
  UpdateExecutionFactory.create(updateRequest, dataset);
updateProcessor.execute();
dataset.commit();

Create READ Transaction and Read

dataset.begin(ReadWrite.READ);
QueryExecution qe = QueryExecutionFactory
  .create("SELECT ?s ?p ?o WHERE {?s ?p ?o .}", dataset);
for (ResultSet results = qe.execSelect(); results.hasNext();) {
  QuerySolution qs = results.next();
  String strValue = qs.get("?o").toString();
  logger.trace("value = " + strValue);
}

Release Dataset Resources and Run Application

The dataset resources can be release calling close() on the dataset.

dataset.close();

Running the application will cause a /db directory to be created in the directory from where you run your application, which consists of the various files that represent your dataset.

Conclusion

In this post I have given a simple example creating a TDB2 dataset and writing to and reading from it. This code can be found on github.

Creating a Remote Repository for GraphDB with RDF4J Programmatically

Posted on April 9, 2018 by Henriette Harmse

In my previous post I have detailed how you can create a local Ontotext GraphDB repository using RDF4J. I indicated that there are some problems when creating a local repository. Therefore, in this post I will detail how to create a remote Ontotext GraphDB repository using RDF4J. As with creating a local repository, there are three steps:

Create a configuration file, which is as for local repositories.
Create pom.xml file, which is as for local repositories.
Create the Java code.

The benefit of creating a remote repository is that it will be under the control of the Ontotext GraphDB Workbench. Hence, you will be able to monitor your repository from the Workbench.

Java Code

package org.graphdb.rdf4j.tutorial;

import java.io.FileInputStream;
import java.io.InputStream;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Iterator;

import org.eclipse.rdf4j.model.Model;
import org.eclipse.rdf4j.model.Resource;
import org.eclipse.rdf4j.model.Statement;
import org.eclipse.rdf4j.model.impl.TreeModel;
import org.eclipse.rdf4j.model.util.Models;
import org.eclipse.rdf4j.model.vocabulary.RDF;
import org.eclipse.rdf4j.repository.Repository;
import org.eclipse.rdf4j.repository.RepositoryConnection;
import org.eclipse.rdf4j.repository.config.RepositoryConfig;
import org.eclipse.rdf4j.repository.config.RepositoryConfigSchema;
import org.eclipse.rdf4j.repository.http.config.HTTPRepositoryConfig;
import org.eclipse.rdf4j.repository.manager.RemoteRepositoryManager;
import org.eclipse.rdf4j.repository.manager.RepositoryManager;
import org.eclipse.rdf4j.repository.manager.RepositoryProvider;
import org.eclipse.rdf4j.rio.RDFFormat;
import org.eclipse.rdf4j.rio.RDFParser;
import org.eclipse.rdf4j.rio.Rio;
import org.eclipse.rdf4j.rio.helpers.StatementCollector;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.Marker;
import org.slf4j.MarkerFactory;

public class CreateRemoteRepository {
  private static Logger logger = LoggerFactory.getLogger(CreateRemoteRepository.class);
  // Why This Failure marker
  private static final Marker WTF_MARKER = MarkerFactory.getMarker("WTF");
	
  public static void main(String[] args) {
    try {		
      Path path = Paths.get(".").toAbsolutePath().normalize();
      String strRepositoryConfig = path.toFile().getAbsolutePath() + "/src/main/resources/repo-defaults.ttl";
      String strServerUrl = "http://localhost:7200";
		
      // Instantiate a local repository manager and initialize it
      RepositoryManager repositoryManager  = RepositoryProvider.getRepositoryManager(strServerUrl);
      repositoryManager.initialize();
      repositoryManager.getAllRepositories();

      // Instantiate a repository graph model
      TreeModel graph = new TreeModel();

      // Read repository configuration file
      InputStream config = new FileInputStream(strRepositoryConfig);
      RDFParser rdfParser = Rio.createParser(RDFFormat.TURTLE);
      rdfParser.setRDFHandler(new StatementCollector(graph));
      rdfParser.parse(config, RepositoryConfigSchema.NAMESPACE);
      config.close();

      // Retrieve the repository node as a resource
      Resource repositoryNode =  Models.subject(graph
        .filter(null, RDF.TYPE, RepositoryConfigSchema.REPOSITORY))
        .orElseThrow(() -> new RuntimeException(
            "Oops, no <http://www.openrdf.org/config/repository#> subject found!"));

		
      // Create a repository configuration object and add it to the repositoryManager		
      RepositoryConfig repositoryConfig = RepositoryConfig.create(graph, repositoryNode);
      repositoryManager.addRepositoryConfig(repositoryConfig);

      // Get the repository from repository manager, note the repository id 
      // set in configuration .ttl file
      Repository repository = repositoryManager.getRepository("graphdb-repo");

      // Open a connection to this repository
      RepositoryConnection repositoryConnection = repository.getConnection();

      // ... use the repository

      // Shutdown connection, repository and manager
      repositoryConnection.close();
      repository.shutDown();
      repositoryManager.shutDown();					
   } catch (Throwable t) {
     logger.error(WTF_MARKER, t.getMessage(), t);
   }		
  }
}

Conclusion

In this post I detailed how you can create remote repository for Ontotext GraphDB using RDF4J, as well as the benefit of creating a remote repository rather than a local repository. You can find the complete code of this example on github.

Creating a Local Repository for GraphDB with RDF4J Programmatically

Posted on April 9, 2018 by Henriette Harmse

If you want to create a local repository for Ontotext GraphDB, according to the documentation. The are essentially 3 steps:

Create a configuration file.
Create a pom.xml file.
The Java code.

However, there are reasons why you may not want to do this, which I detail.

Configuration File

@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
@prefix rep: <http://www.openrdf.org/config/repository#>.
@prefix sr: <http://www.openrdf.org/config/repository/sail#>.
@prefix sail: <http://www.openrdf.org/config/sail#>.
@prefix owlim: <http://www.ontotext.com/trree/owlim#>.

[] a rep:Repository ;
  rep:repositoryID "graphdb-repo" ;
  rdfs:label "graphdb-repo-label" ;
  rep:repositoryImpl [
    rep:repositoryType "graphdb:FreeSailRepository" ;
    rep:repositoryType "owlim:MonitorRepository" ;
    sr:sailImpl [
      sail:sailType "graphdb:FreeSail" ;
       
      owlim:base-URL "http://myexample.ontotext.com/graphdb#" ;
      owlim:defaultNS "" ;
      owlim:entity-index-size "10000000" ;
      owlim:entity-id-size  "32" ;
      owlim:imports "" ;
      owlim:repository-type "file-repository" ;
      owlim:ruleset "owl-horst-optimized" ;
      owlim:storage-folder "storage" ;
  
      owlim:enable-context-index "true" ;
      owlim:cache-memory "256m" ;
      owlim:tuple-index-memory "224m" ;

      owlim:enablePredicateList "true" ;
      owlim:predicate-memory "32m" ;

      owlim:fts-memory "0" ;
      owlim:ftsIndexPolicy "never" ;
      owlim:ftsLiteralsOnly "true" ;

      owlim:in-memory-literal-properties "true" ;
      owlim:enable-literal-index "true" ;
      owlim:index-compression-ratio "-1" ;
           
      owlim:check-for-inconsistencies "false" ;
      owlim:disable-sameAs "false" ;
      owlim:enable-optimization "true" ;
      owlim:transaction-mode "safe" ;
      owlim:transaction-isolation "true" ;
      owlim:query-timeout "0" ;
      owlim:query-limit-results "0" ;
      owlim:throw-QueryEvaluationException-on-timeout "false" ;
      owlim:useShutdownHooks "true" ;
      owlim:read-only "false" ;
    ]
  ].

`pom.xml` File

   
   <dependency>
      <groupId>com.ontotext.graphdb</groupId>
      <artifactId>graphdb-free-runtime</artifactId>
      <version>8.4.1</version>
   </dependency>

Java Code

package org.graphdb.rdf4j.tutorial;

import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.nio.file.Path;
import java.nio.file.Paths;

import org.eclipse.rdf4j.model.Resource;
import org.eclipse.rdf4j.model.impl.TreeModel;
import org.eclipse.rdf4j.model.util.Models;
import org.eclipse.rdf4j.model.vocabulary.RDF;
import org.eclipse.rdf4j.repository.Repository;
import org.eclipse.rdf4j.repository.RepositoryConnection;
import org.eclipse.rdf4j.repository.config.RepositoryConfig;
import org.eclipse.rdf4j.repository.config.RepositoryConfigSchema;
import org.eclipse.rdf4j.repository.manager.LocalRepositoryManager;
import org.eclipse.rdf4j.repository.manager.RepositoryManager;
import org.eclipse.rdf4j.rio.RDFFormat;
import org.eclipse.rdf4j.rio.RDFParser;
import org.eclipse.rdf4j.rio.Rio;
import org.eclipse.rdf4j.rio.helpers.StatementCollector;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.Marker;
import org.slf4j.MarkerFactory;

public class CreateLocalRepository {
  private static Logger logger = LoggerFactory.getLogger(CreateLocalRepository.class);
  // Why This Failure marker
  private static final Marker WTF_MARKER = MarkerFactory.getMarker("WTF");
	
  public static void main(String[] args) {
    try {		
      Path path = Paths.get(".").toAbsolutePath().normalize();
      String strRepositoryConfig = path.toFile().getAbsolutePath() + 
          "/src/main/resources/repo-defaults.ttl";
		
      // Instantiate a local repository manager and initialize it
      RepositoryManager repositoryManager = new LocalRepositoryManager(new File("."));
      repositoryManager.initialize();

      // Instantiate a repository graph model
      TreeModel graph = new TreeModel();

      // Read repository configuration file
      InputStream config = new FileInputStream(strRepositoryConfig);
      RDFParser rdfParser = Rio.createParser(RDFFormat.TURTLE);
      rdfParser.setRDFHandler(new StatementCollector(graph));
      rdfParser.parse(config, RepositoryConfigSchema.NAMESPACE);
      config.close();

      // Retrieve the repository node as a resource
      Resource repositoryNode =  Models.subject(graph
         .filter(null, RDF.TYPE, RepositoryConfigSchema.REPOSITORY))
         .orElseThrow(() -> new RuntimeException(
             "Oops, no <http://www.openrdf.org/config/repository#> subject found!"));

      // Create a repository configuration object and add it to the repositoryManager
      RepositoryConfig repositoryConfig = RepositoryConfig.create(graph, repositoryNode);
      repositoryManager.addRepositoryConfig(repositoryConfig);

      // Get the repository from repository manager, note the repository id
      // set in configuration .ttl file
      Repository repository = repositoryManager.getRepository("graphdb-repo");

      // Open a connection to this repository
      RepositoryConnection repositoryConnection = repository.getConnection();

      // ... use the repository

      // Shutdown connection, repository and manager
      repositoryConnection.close();
      repository.shutDown();
      repositoryManager.shutDown();					
    } catch (Throwable t) {
      logger.error(WTF_MARKER, t.getMessage(), t);
    }		
  }
}

Why you may not want to do this

new LocalRepositoryManager(new File(".")); will create a repository where ever your Java application is running from. This means the repository will not be under the control of your Ontotext GraphDB Workbench. Hence, you will not be able to run SPARQL queries or monitor your database from the Workbench. I am not aware of any way via which you can instruct GraphDB to look for repositories in an additional directory.

If you change the directory to $GRAPH DB INSTALL$/data/repositories, the repository will be under the control of Ontotext GraphDB (assuming you have a local GraphDB instance) only if GraphDB is not running. If you start GraphDB after running your program, you will be able to see the repository in GraphDB workbench.

Conclusion

In this post I have detailed how you can create an Ontext GraphDB repository using RDF4J and why you may not want to do this. In my next post I detail how
to create a remote repository, which addresses the problem I detailed here. You can find the complete code of this example on github.

Henriette Harmse

Linked Data, Knowledge Graphs, Semantic Web and Art

Category Archives: Uncategorized

Creating, Writing and Reading Jena TDB2 Datasets

Create TDB2 Dataset

Create WRITE Transaction and Write

Create READ Transaction and Read

Release Dataset Resources and Run Application

Conclusion

Creating a Remote Repository for GraphDB with RDF4J Programmatically

Java Code

Conclusion

Creating a Local Repository for GraphDB with RDF4J Programmatically

Configuration File

`pom.xml` File

Java Code

Why you may not want to do this

Conclusion

Create TDB2 Dataset

Create WRITE Transaction and Write

Create READ Transaction and Read

Release Dataset Resources and Run Application

Conclusion

Java Code

Conclusion

Configuration File

pom.xml File

Java Code

Why you may not want to do this

Conclusion

`pom.xml` File