Technical resources

Introductory Videos

  • (2020, most of these capabilities are in GA now - September 2022)

AusPIX Spatially Linked-data APIs

AusPIX– providing metrics and spatial statistics (using a pre-calculated relationship grid built on DGGS technology)

(swagger page with many tools)

This API provides cross-referencing and statistical tools across geographical features. Exposes relationships between datasets to provide quick metrics and spatial statistics. Both human and machine to machine (automated) queries are easily processed, with results in json format. AusPIX handles points, lines, polygons and rasters.

:

: Discover the features recorded at a latitude/ longitude point. Useful for any query to discover the information at a given location. Very quick. Can answer such questions as:

  1. What is the name of the Electorate/ SA2/ LGA / Suburb / postcode / National Park that the location within?
  2. What is the risk related to earthquakes or cyclones at that location?
  3. Is the location likely to be ever flooded? (using WofS data)
  4. I have a list for Greater Glider possum sightings with latitude and longitude information. What is the approximate elevation above sea-level for these possum sightings?
  5. I need situational awareness for school locations and addresses. This tool allows me to discover which Electorate/ SA2/ LGA / Suburb / postcode etc a school or street address is located in.
  6. Supply this sort of information to higher level dashboards and APIs (via machine readable API calls using simple latitude and longitude parameters)

: discover the features and data available for a DGGS cell. Similar to the Data drill tool, returning information for a query on a DGGS AusPIX level 10 cell.

: Discover the spatial relationship between a single feature in one geography and the features of any other geography. Quick analysis. User chooses the cross-analysis information they are looking for from countless options. Response is as a json string and includes a range of statistical results. Typical use cases include.

  1. I have an LGA. Tell me which SA1, SA2, SA3 etc, Postcodes, Suburbs, National Parks, Water Bodies, Streams, roads, Schools, Place names, Emergency Facilities etc etc are within or partly within (with % for apportionment).
  2. I am looking to buy a house and want to know the suburbs and the schools in them so I can choose. The user or a connected API can query suburbs for a list of schools in each one.
  3. I have a flood polygon that has been DGGS enabled and want some statistics on what land-uses have been affected within the flooded area. The user uses AusPIX to cross-reference the flooded area with land-use data that is already in the the AusPIX system.
  4. A bushfire has swept across large areas. The minister wants to have a list of the suburbs/postcodes/electorates that have been burnt along with the % figures to support a recovery initiatives.

: I have an area of interest as a WKT (well-known text) polygon. I need AusPIX cross-referenced information to describe and inform me about what features exist within that polygon.  The API provides apportionment information for this WKT user-defined polygon. For example, the WKT polygon may represent the landfall of a cyclone and the Emergency Service wants information about what is threatened in the impact zone. The AusPIX system can quickly cross-reference the WKT polygon with any geographic data available in the AusPIX system.

Other uses: There are literally hundreds of use-cases involving cross-referencing. AusPIX can support crosswalks between any data that has been DGGS enabled. No need to wait for reprocessing, the AusPIX system based on DGGS cells includes all combinations, being able to relate to any combination.

Other AusPIX DGGS based APIs

  • (AusPIX DGGS API tool to DGGS enable small-medium datasets (human and machine readable)
  • Demonstrator to build your own AusPIX DGGS spatial reference for points, lines or polygons.
  • (Provides human-readable and machine-readable access to cell information including Landing Page with map showing cell location and data describing location of cell corners and cell centroid. Parent cells and child cells are described.)

Semantically Linked data APIs

Providing for semantic linked-data queries, metadata information, provenance information, and individual feature mapping. These APIs source their information from RDF triple stores containing a copy of the source data. Features of these APIs are:

  • standardised interoperability
  • human readable landing page for each geographic feature,
  • machine readability for automation,
  • variety of alternate profiles for viewing and accessing data
  • metadata, conformance, and vocab information
  • searchable
  • built for semantic queries
  • cross dataset queries need to be done within the same triple store

Semantically linked data cannot natively provide apportionment figures. To work around this semantic systems sometime use different types of spatially linked databases (like PostGIS) and do spatial calculations to provide these apportionment figures (% relationships between features in different datasets). GeoSPARQL can also provide some of these calculations but the time takes is similar to a GIS package calculation doing the same thing. The AusPIX spatially linked APIs fill this gap to provide greater speed and flexibility.

Because these systems are built as machine-readable APIs, the network of information is available to API calls to gather exactly the information that is required.

Use cases for semantically linked-data using APIs

  1. A user has identified a feature (e.g an LGA) and needs to see it on a map, discover its attributes, and determine the age & provenance of the information. The search feature in the API allows this user to find the LGA and provide the information required. Note: the user has feature level access and doesn’t need to download and manipulate the whole dataset.
  2. A user quickly wants the geometry for Commonwealth Electorate. The user finds the Electorate in the Linked Data API, and copies the geometry (as geojson) to a map.
  3. A data scientist wants to semantically query Wikidata information against some GA Triple Store of RDF data. The scientist imports the required Wikidata triples and the GA/ABS RDF triples into a Wikibase instance and proceeds to discover trends and relationships for a study on Australian population dynamics. Since Wikidata and GA linked data are stored as RDF triples the schemas are identical – making the job a lot easier.
  4. To make data more usable under a Government directive regarding F.A.I.R principles, 女女视频publishes its data as web-connected RDF triples. This allows the GA data to be part of systems that connect and query data from different sources. The RDF triples provide a common schema for rapid and standardised transfer of information. APIs written over the RDF triple stores allow diverse range of dashboards and informative web pages to be developed for special use cases. GA provides the data as Triples, builds some useful APIs inside GA, and allows others to build and maintain their own APIs over the GA world standard RDF data sources.

Available Semantic Linked-Data APIs in GA

  • - Electrical Infrastructure, Emergency Facilities, Floods, Placenames
  • in conjunction with ABS
  • Geofabric catchments, divisions and river regions
  • (GNAF)
  • for loading datasets to be cross-referenced, data will need to be added and removed as required. This is where data is loaded for temporary cross-referencing work. (NB as the technology stands today, for cross-referencing to work all relevant RDF triples must be located in the same RDF Triple Store).

Loci API products adapted for and hosted in Geoscience Australia

  • built over 女女视频Triple stores - an interactive feature discovery tool – (not working within GA, but working fine externally)
  • – code to build “Explorer”
  • - use this code to run locally in Jupyter Notebook – a tool to find ID’s of geographical features using RDF triple stores and data source.
  • API has a SPARQL endpoint so that SPARQL can be used to query the database. Sometime CQL or other query methods are provided as alternative query languages. Examples are provided including examples of how machine-readable queries are built.

Technical Documentation

  • by Surround Australia Pty Ltd for Geoscience Australia
  • Loci notebooks (previous development – AusPIX DGGS integration with Semantic Linked data included)
  • (older development information)
  • (Loc-I initial project stakeholder engagement)

Training Plans

  • -Surround Digital Atlas Linked Data training plan for loading and modifying datasets.

Ontologies

Concepts and categories with their properties and the relationships between them - providing data models and vocabularies.

Vocabulary Documentation and Tools

  • vocabulary reader tool - human and machine readable
  • - interact with SPARQL queries
  • Adds the ASGS, GA, and GNAF themes to a Prez docker image. NOTE: Bitbucket resources require access authorisation

Provenance development

Semantic Validation tools

  • – Dashboard user interface using pySHACL - to suit many styles of RDF.

Code Repositories GitHub and Bitbucket

NOTE: Bitbucket resources require access authorisation.

  • (on PyPi)
  • - Code to build and query AusPIX DGGS Crosswalk Tables.
  • DGGS
  • (The GitHub repository for the code for Placenames Linked Data API)
  • (built on DGGS level 4 cells, approximately 140km by 140km tiles to suit multi-CPU processing)
  • RDF Linked Datasets available through their Linked-Data APIs (with landing page for each feature, human readable and machine readable)
  • (our original copy in 女女视频infrastructure)
  • (Built from AusPIX crosswalk tables. Tableau Software can access the AWS PostgreSQL AusPIX crosswalk data to display information in dashboards) Tableau cannot be automated to the same extent as our AusPIX or Semantic workflows.