Home » Articles posted by Henriette Harmse
Author Archives: Henriette Harmse
Using SHACL validation with Ontotext GraphDB
Today I have 1 of those moments where I am absolutely sure if I do not write this down, I will forget how to do this next time. For one of the projects I am working on, we need to do SHACL validation of RDF data that will be stored in Ontotext GraphDB. Here are the 10 things I needed to learn in doing this. Some of these are rather obvious, but some were less than obvious to me.
Number 1: To be able to do SHACL validation, your repository needs to be configured for SHACL when you create your repository. This cannot be done after the fact.
Number 2: It seems to be better to import your ontology (or ontologies) and data into different graphs. This is useful when you want to re-import your ontology (or ontologies) or your data, because then you can replace a specific named graph completely. This was very useful for me while prototyping. Screenshot below:

Number 3: SHACL shapes are imported into this named graph
http://rdf4j.org/schema/rdf4j#SHACLShapeGraph
by default. At configuration time you can provide a different named graph or graphs for your SHACL shapes.
Number 4: To find the named graphs in your repository, you can do the following SPARQL query:
select distinct ?g
where {
graph ?g {?s ?p ?o }
}
You can then query a specific named graph as follows:
select *
from <myNamedGraph>
where {
?s ?p ?o .
}
Number 5: However, getting the named graphs does not return the SHACL named graph. On StackOverflow someone suggested SHACL shapes can be retrieved using:
http://address:7200/repositories/myRepo/rdf-graphs/service?graph=http://rdf4j.org/schema/rdf4j#SHACLShapeGraph
However, this did not work for me. Instead, the following code worked reliably:
import org.eclipse.rdf4j.model.Model;
import org.eclipse.rdf4j.model.impl.LinkedHashModel;
import org.eclipse.rdf4j.model.vocabulary.RDF4J;
import org.eclipse.rdf4j.repository.RepositoryConnection;
import org.eclipse.rdf4j.repository.http.HTTPRepository;
import org.eclipse.rdf4j.rio.RDFFormat;
import org.eclipse.rdf4j.rio.Rio;
import org.eclipse.rdf4j.rio.WriterConfig;
import org.eclipse.rdf4j.rio.helpers.BasicWriterSettings;
import java.util.stream.Collectors;
public class RetrieveShaclShapes {
public static void main(String[] args) {
String address = args[0]; /* i.e. http://localhost/ */
String repositoryName = args[1]; /* i.e. myRepo */
HTTPRepository repository = new HTTPRepository(address, repositoryName);
try (RepositoryConnection connection = repository.getConnection()) {
Model statementsCollector = new LinkedHashModel(
connection.getStatements(null, null,null, RDF4J.SHACL_SHAPE_GRAPH)
.stream()
.collect(Collectors.toList()));
Rio.write(statementsCollector, System.out, RDFFormat.TURTLE, new WriterConfig().set(
BasicWriterSettings.INLINE_BLANK_NODES, true));
} catch (Throwable t) {
t.printStackTrace();
}
}
}
using the following dependencies in the pom.xml
with
${rdf4j.version} = 4.2.3:
<dependency>
<groupId>org.eclipse.rdf4j</groupId>
<artifactId>rdf4j-client</artifactId>
<version>${rdf4j.version}</version>
<type>pom</type>
</dependency>
Number 6: Getting the above code to run was not obvious since I opted to using a fat jar. I encountered an “org.eclipse.rdf4j.rio.UnsupportedRDFormatException: Did not recognise RDF format object
” error. RFD4J uses the Java Service Provider Interface (SPI) which uses a file in the META-INF/services
of the jar to register parser implementations. The maven-assembly-plugin
I used, to generate the fat jar, causes different jars to overwrite META-INF/services
thereby loosing registration information. The solution is to use the maven-shade-plugin which merge META-INF/services
rather overwrite it. In your pom you need to add the following to your plugins configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.4.1</version>
<executions>
<execution>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
You can avoid this problem by using the separate jars rather than a single fat jar.
Number 7: Importing a new shape into the SHACL shape graph will cause new shape information to be appended. It will not replace the existing graph even when you have both the
- “Enable replacement of existing data” and
- “I understand that data in the replaced graphs will be cleared before importing new data.”
options enabled as seen in the next screenshot:

To replace the SHACL named graph you need to clear it explicitly by running the following SPARQL command:
clear graph <http://rdf4j.org/schema/rdf4j#SHACLShapeGraph>
For myself I found it easier to update the SHACL shapes programmatically. Note that I made use of the default SHACL named graph:
import org.eclipse.rdf4j.model.vocabulary.RDF4J;
import org.eclipse.rdf4j.repository.RepositoryConnection;
import org.eclipse.rdf4j.repository.http.HTTPRepository;
import org.eclipse.rdf4j.rio.RDFFormat;
import java.io.File;
public class UpdateShacl {
public static void main(String[] args) {
String address = args[0]; /* i.e. http://localhost/ */
String repositoryName = args[1]; /* i.e. myRepo */
String shacl = args[2];
File shaclFile = new File(shacl);
HTTPRepository repository = new HTTPRepository(address, repositoryName);
try (RepositoryConnection connection = repository.getConnection()) {
connection.begin();
connection.clear(RDF4J.SHACL_SHAPE_GRAPH);
connection.add(shaclFile, RDFFormat.TURTLE, RDF4J.SHACL_SHAPE_GRAPH);
connection.commit();
} catch (Throwable t) {
t.printStackTrace();
}
}
}
Number 8: Programmatically you can delete a named graph using this code and the same maven dependency as we used above:
import org.eclipse.rdf4j.model.IRI;
import org.eclipse.rdf4j.model.ValueFactory;
import org.eclipse.rdf4j.model.impl.SimpleValueFactory;
import org.eclipse.rdf4j.repository.RepositoryConnection;
import org.eclipse.rdf4j.repository.http.HTTPRepository;
public class ClearGraph {
public static void main(String[] args) {
String address = args[0]; /* i.e. http://localhost/ */
String repositoryName = args[1]; /* i.e. myRepo */
String graph = args[2]; /* i.e. http://rdf4j.org/schema/rdf4j#SHACLShapeGraph */
ValueFactory valueFactory = SimpleValueFactory.getInstance();
IRI graphIRI = valueFactory.createIRI(graph);
HTTPRepository repository = new HTTPRepository(address, repositoryName);
try (RepositoryConnection connection = repository.getConnection()) {
connection.begin();
connection.clear(graphIRI);
connection.commit();
}
}
}
Number 9: If you update the shape graph with constraints that are violated by your existing data, you will need to first fix your data before you can upload your new shape definition.
Number 10: When uploading SHACL shapes, unsupported features fails silently. I had this idea to add human readable information to the shape definition to make it easier for users to understand validation errors. Unfortunately “sh:name
” and “sh:description
” are not supported by GraphDB version 10.0.2. and 10.2.0. Moreover, it fails silently. In the Workbench it will show that it loaded successfully as seen in the next screenshot:

However, in the logs I have noticed the following warnings:

As these are logged as warnings, I was expecting my shape to have loaded fine, except that triples pertaining to “sh:name
” and “sh:description
” are skipped. However, my shape did not load at all.
You find the list of supported SHACL features here.
Conclusion
This post may come across as being critical of GraphDB. However, this is not the intention. I think it is rather a case of growing pains that are still experienced around SHACL (and Shex, I suspect) adoption. Resources that have been helpful for me in resolving issues are:
- GraphDB documentation, and
- RDF4J on which GraphDB is built.
Introduction to ontology semantics and reasoning
I recently had the pleasure to present at the OntoSpot meeting at EBI to help my colleagues gain an intuitive understanding of ontology semantics and reasoning. In this talk I assume that you have a very basic understanding of what an ontology is, but I assume no previous knowledge wrt logic. I provide a number of examples and graphics to explain logic and description logic (DL) concepts.
Here I provide both the slides of this presentation and the link to the recording. If you have any questions or suggestions, please let me know in the comments. I have already had the very helpful suggestion for adding a reference of DL symbols, which I will do shortly.
Errata:
- In the section on speaking about propositional logic, I accidentally said predicate logic instead of propositional logic.
- At the end while answering questions, I said RFD rather than RDF.
This video will also be made available at the OBO Academy.
Biological ontology use guidelines
This short guideline aims to
- Explain the difference between a domain ontology versus an application ontology;
- Provide some guidelines for choosing a single ontology or a small set of ontologies;
- Explain when to consider creating a new domain ontology; and
- Give guidance as to when to create an application ontology.
An upfront disclaimer: I am working at EBI and hence those are the tools I am most familiar with. I have tried where possible to include tools from the biological/biomedical community outside EBI. But I might have missed some. Moreover, this post is not about punting any tools. Rather, it is to describe a general guideline wrt choosing ontologies.
Domain ontologies versus application ontologies
A domain ontology has terms that only concern a single well defined domain and aims to address the general requirements of the community of that domain. It is my understanding that domain ontologies adhere to Principle 5 of the OBO principles. The Data Use Ontology (DUO) is an example of a domain ontology.
An application ontology tends to serve the specific needs for a specific application and often spans different domains, though it may deal with a single domain. The Experimental Factor Ontology (EFO) is an example of an application ontology.
Guidelines for choosing an ontology or ontologies
Here some basic guidelines are given for choosing an ontology.
- Evaluate potential target ontologies using the guidelines of Ten Simple Rules for Selecting a Bio-ontology and Which biomedical ontologies should we use? Potential target ontologies can be found using tools like the Ontology Lookup Service (OLS), Ontobee and Bioportal. The Open Biological and Biomedical Ontology (OBO) Foundry is a community driven effort to create interoperable biological ontologies. It is therefore a good starting point for identifying viable ontologies that are used and supported by an active community.
- Try to identify 1 ontology or as few as possible target ontologies that may address your application needs. This can be done using a tool like Zooma to map strings of text to terms in ontologies based on manual curated data or some machine learning algorithm. Based on these terms a single ontology or multiple target ontologies can be identified.
- Try to map terms from less preferred ontologies to terms in preferred target ontologies using a tool like OxO. If such mappings cannot be found, but to the best of your knowledge term A in ontology A should map to term B in ontology B, consider opening tickets against both ontology A and ontology B to have the respective mappings added. It is only by the community actively contributing to these ontologies that the full FAIR value proposition of ontologies are realised.
- As far as possible try to use or contribute to existing ontologies rather than introduce your own ontology. This means when an ontology fits your needs reasonably well, but it is missing some terms required by your use case, it is best to collaborate with the relevant ontology designers to try and add the missing terms. For this purpose most of the ontologies in the biological and biomedical community make use of GitHub Issue trackers where you can open new issues, give your input to existing issues and get latest release information. Remember, the main value proposition of ontologies wrt FAIR principles is realised exactly when an ontology is used, reused and extended as part of a community that has some shared objectives.
- Try to keep the number of ontologies you want to use as small as is sensible (that is the smallest set of ontologies that are well aligned with the needs of your use case). The reason for this is that you will want to engage with the designers of the ontologies you use to extend and amend these ontologies for your use case. The more ontologies you use, the chances are that you will need to communicate with a larger community to affect change for your use case. This may increase development times. However, using an ontology that is not well aligned with your use case will also increase communication and timelines. Thus, the reason for keeping the number of ontologies as small as is sensible.
When to create a domain ontology
If after a review of the existing ontologies you find that your domain is not covered by any of the existing ontologies, you have a strong case for creating your own ontology. At this point you can decide to create the ontology yourself from scratch. When you are just starting down the path of creating your own ontology, it may seem quicker and easier than to have external collaborators involved. However, if you can create your ontology in collaboration with ontology designers that have a deep understanding of the creation and long term maintenance of biological ontologies, it will help you to avoid various pitfalls in ontology creation. Such collaborations can seem long-winded, but will save a lot of time and effort in the long run. A key point to keep in mind when designing an ontology is not only to be concerned with the initial creation of the ontology, but also with its long term maintainability.
When to create an application ontology
Only consider introducing your own application ontology in the following cases:
- Your use case spans a number of domains that are served by subsets of terms coming from a number of ontologies. In this case the application typically uses much smaller subsets of terms from the original ontologies. For this scenario an application ontology can be introduced that imports the relevant terms from existing ontologies. An example of this is EFO that is used by a number of teams in EBI. Again, as far as possible terms should be used from existing ontologies rather than introducing terms in your own ontology (see 4 above). Note that when a use case spans a number of domains served by a number of ontologies and the number of terms used from the ontologies are very close to all terms in the ontologies, it is best to use the ontologies directly rather than to introduce your own application ontology.
- When you have application specific needs that cannot be addressed by any existing domain or application ontology, then only it may make sense to create your own application ontology.
Final thoughts
- As far as possible give preference to re-using ontologies rather than introducing you own.
- When you want to embark on creating either a domain or application ontology, I really strongly recommend reaching out the OBO Foundry. They have been developing biological and biomedical ontologies for years now, which means they have learned a lot of lessons over the years. By speaking to them it can help you avoid the typical problems people tend to run into when trying to create their first ontology.