Matching VOEvent and Characterisation Files with SPARQL

Science Case

The IVOA has defined standards for VOEvent and Characterisation XML files. Both of these standards reference the IVOA's Space-Time Coordinate Metadata standard. VOEvents describe space events such as supernovae, GRBs, solar flares, etc. The right ascension and declination coordinates of astronomical events are defined in the VOEvent WhereWhen element using the STC AstroCoords elements. Characterisation files describe observational data. An instrumental field of view can be characterised with spatial area coverage using the STC Polygon elements, and each polygon is described by n vertices.

Problem: Determine whether an event described by a VOEvent file appears in the field of view of an observation described by a Characterisation file.

Solution: Extract right ascension and declination from VOEvent file, then use the Java polygon class contains() method to determine whether the VOEvent coordinate pair lies within Characterisation field-of-view polygon.

Input Data

VOEvents

Events from three existing catalogues were converted to VOEvent packets to use as test data. In addition, event metadata was uploaded to a MySQL^? database at MSSL, and a STAP service was deployed for each catalogue following the procedure outlined at http://wiki.astrogrid.org/bin/view/Astrogrid/VoEventSTAPtutorial. These catalogues were chosen because a sizable portion of event start times overlap with the 2MASS data set.

• SAI supernova catalogue: 1013 files (1997 - 2001)
• Asiago supernova catalogue: 4063 files (1885 - 2007)
• OGLE II DIA microlensing event catalogue: 521 files(1997 - 2000)

Characterisation

Francois Bonnarel provided characterisation files describing Two Micron All Sky Survey (2MASS) near-infrared observations covering a region of sky between RA: 10 00 00 to 12 47 34 and Dec: -00 50 00 to 00 50 00.

• 2MASS H band: 9980 files
• 2MASS J band: 10253 files
• 2MASS K band: 10125 files

Procedure

Convert XML to RDF

First, VOEvent and Characterisation XML files were converted to RDF. Once a valid RDF template had been established for each file type using the W3C RDF Validation Service, the original XML was converted to RDF using an XSLT stylesheet and the xsltproc tool.

Perform SPARQL Queries

Next, two SPARQL queries were constructed using RDF attributes as well as relationships from the VoEventOntology and CharacterisationOntology. Query voevent_query.rq extracts RA, Dec, and VOEvent packet URLs from VOEvent RDF data, while char_query.rq extracts polygon vertices and Characterisation file URLs from Characterisation RDF data:

voevent_query.rq:

PREFIX charo: <http://eurovotech.org/twiki/bin/viewfile/VOTech/CharacterisationOntology?rev=1;filename=characterisation1.11.owl>
PREFIX stco: <http://wiki.eurovotech.org/twiki/bin/viewfile/VOTech/StcOntology?rev=3;filename=STC1.3_II.owl>
PREFIX voeo: <http://wiki.eurovotech.org/twiki/bin/viewfile/VOTech/VoEventOntology?rev=1;filename=VOEvent1.1.owl>
SELECT ?C1 ?C2 ?url
FROM
WHERE {
?voevent voeo:hasWhereWhen ?WhereWhen .
?WhereWhen voeo:hasObsDataLocation ?ObsDataLocation .
?ObsDataLocation stco:hasObservationLocation ?ObservationLocation .
?ObservationLocation stco:hasAstroCoords ?AstroCoords .
?AstroCoords stco:hasPosition2D ?Position2D .
?Position2D stco:hasValue2 ?Value2 .
?Value2 stco:C1 ?C1 .
?Value2 stco:C2 ?C2 .
?url voeo:role "observation" .
}

char_query.rq:

PREFIX charo: <http://eurovotech.org/twiki/bin/viewfile/VOTech/CharacterisationOntology?rev=1;filename=characterisation1.11.owl>
PREFIX stco: <http://wiki.eurovotech.org/twiki/bin/viewfile/VOTech/StcOntology?rev=3;filename=STC1.3_II.owl>
SELECT distinct ?C1 ?C2 ?url
FROM
WHERE {
?char charo:hascharacterizationAxis ?axis .
?axis charo:hascoverage ?coverage .
?coverage charo:hassupport ?support .
?support charo:AreaType ?AreaType .
FILTER regex(?AreaType, "Polygon")
?support charo:hasArea ?Area .
?Area stco:hasPolygon ?Polygon .
?Polygon stco:hasVertex ?Vertex .
?Vertex stco:hasPosition ?Position .
?Position stco:C1 ?C1 .
?Position stco:C2 ?C2 .
?url charo:hascharacterizationAxis ?axis .
}

SPARQL queries were tested on individual VOEvent and Characterisation files using the ARQ 2.1 command line query interface:

bash-2.05b$ sparql --query voevent_query.rq
------------------------------------------------------------------------------------------
| C1 | C2 | url |
==========================================================================================
| "36.8783" | "+2.68528" | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/rdf/SAI/SAI2001hj.rdf> |
------------------------------------------------------------------------------------------

-bash-2.05b$ sparql --query char_query.rq
---------------------------------------------------------------------------------------------------------------
| C1 | C2 | url |
===============================================================================================================
| "\n 194.260654\n " | "\n -0.884861\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
| "\n 194.402896\n " | "\n -0.742643\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
| "\n 194.402894\n " | "\n -0.884865\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
| "\n 194.331783\n " | "\n -0.600419\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
| "\n 194.260668\n " | "\n -0.600416\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
| "\n 194.402898\n " | "\n -0.600420\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
| "\n 194.260661\n " | "\n -0.742638\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
| "\n 194.331774\n " | "\n -0.884863\n " | <http://msslxt.mssl.ucl.ac.uk:8080/voevent/xml/CharData/SIA_194.31+-0.71_2MASS_H_990211S_HI0880233> |
---------------------------------------------------------------------------------------------------------------

Test Coordinates Inside Polygons

Once the SPARQL queries had been tested, the ARQ 2.1 Java libraries were used to iterate through directories of VOEvent and Characterisation files, perform the SPARQL queries, reformat query results as Java Polygons, doubles, and Strings, and finally to use the Java Polygon contains() method to identify VOEvent coordinate pairs located within Characterisation FOV polygons.

Where coordinates and polygons were found to match, the resulting output returned the original VOEvent packet URL, VOEvent right ascension and declination, and the original Characterisation XML file URL. Initial tests using the upper left / lower right bounds of Characterisation file coverage suggested 28 potential matching events. However, only 7 VOEvents were found to occur within the instrumental field of view for 46 Characterisation files. The 7 events were all from the SAI supernova catalogue, but as this catalogue overlaps with the Asiago supernova catalogue, the Polygon queries should have returned both SAI and Asiago supernova events. Possible factors for the omission of Asiago events include errors in coordinate conversion, problems with skipped files or directories, or close but not identical metadata contained in the SAI and Asiago catalogues.

Matching VOEvent and Characterisation results are below.

Results

VOEvents Matching Characterisation Files

The following 7 VOEvent packets describing supernovae from the SAI catalogue match 2MASS H, J, or K observation fields of view:

Notes on Implementation

The queries used to extract VOEvent coordinates and Characterisation polygon vertices were implemented with SPARQL queries against RDF data. However, similar queries could have been performed using SQL with event and observational metadata stored in RDBMS tables, or with XQuery queries against VOEvent packets and Characterisation XML files stored in XML databases or a file system. The SPARQL queries use relationships defined in the VOEvent, STC, and Characterisation ontologies, so future SPARQL queries against the same RDF datasets could be extended to include inferencing.

The ARQ 2.1 Java SPARQL query interface does not require RDF data to be loaded into a triple store (such as Sesame). Instead, SPARQL queries can be executed over a local file system or remote URLs. A future implementation option to explore would be the use of ARQ to execute SPARQL queries over a list of VOEvent / Characterisation URLs supplied inside a VOTable. This would allow SPARQL queries to be executed against VOEvent and Characterisation RDF data hosted at distributed sites instead of loading the data into a central SPARQL endpoint.

Pros:

1. SPARQL can return coordinates and polygon vertices from RDF as easily as SQL could return such values from an RDBMS or XQuery from XML files.
2. Using ARQ, SPARQL queries can return results from distributed RDF files. No need to load data into a central RDBMS or SPARQL endpoint.
3. These SPARQL queries could be edited to include more sophisticated inferencing against the VOEvent and Characterisation datasets, perhaps using a combination of the VOEvent, STC, and Char. ontologies with the astronomy, physics, and IVOAO ontologies hosted at the University of Maryland. (However, the current queries do not include sophisticated inferencing.)

Cons:

1. SPARQL query results probably slower than comparable SQL query results.
2. VOEvent and Characterisation XML data written to IVOA standards had to be reformatted as RDF.

-- ElizabethAuden - 15 Mar 2008