ARCS Data Services

This page is intended for expert users and developers. For a softer introduction for ordinary users please go to the main ARCS web site

Case studies

Here are some projects currently using ARCS data services

• ILDG
• Skymapper
• AuScope
• LIGO
• CSIRO
• Curtin University
• Australian MODIS Archive
• ANSTO
• Australian Synchrotron
• BlueNet
• IMOS
• Human Variome Project
• MACDDAP

ARCS Data Fabric

Read all about it at the main ARCS web site

Service documentation

• SRB

Other data services

See here for a description of other ARCS Data Services

There's also

• SRM interface. This has more functionality than GridFTP such as reserving storage and can use GridFTP as the underlying transport.

Deployment of data services

For many sites the basic data services would be available in a standard virtual machine called ngdata. The data services would be attached to appropriate storage such as a compute cluster file system, a data repository or an instrument's data cache. Sites may choose to install individual services on existing machines for various reasons.

The centralised services will be only installed in one or two locations ensuring that there is some kind of high availability solution for these services.

Referring to data location from within a grid job

EnvironmentVariables defines how standard environment variables can be used to access data from a grid job. It was written for the APAC program but it is still relevant.

Data Collaboration Services meetings

Slides

• Stephen_McMahon_V5_revised.ppt: eResearch'08 data fabric talk. Updated for accuracy since the conference.
• Data_Fabric_Workshop.ppt: eResearch'08 data fabric workshop slides. Updated for accuracy since the conference.
• Hermes.ppt: Hermes slides thanks to Matthew Wyatt

-- StephenMcMahon - 20 Oct 2008