ASTRO Grid

2005-2007

ASTRO-Grid

This photographic print was digitized as part of the SuperCOSMOS project, creating massive raster libraries referenced by spherical coordinates. Later digital-camera sky surveys such as the Sloan Sky Survey scanned the universe at increasing resolution and sensitivity.

Bespoke data pipelines sucked these pixel images into a process that cleaned them of satellite and camera artifacts and extracted characteristics of stars and galaxies and nebulae. Such pipelines were highly sensitive to configurations and would be re-run with different properties for different end analysis.

As instruments improved and proliferated, data size and complexity rocketed - but internet bandwidth trailed in relative terms. The data was not just 'big' but rapidly growing, poorly described, and difficult to associate across sensors and research domains. Data transfer often had to be physical: data was copied to a disk drive and then the drive or complete PC parcel-posted to international collaborations.

Unusual Systems staff built data publishing, archiving and querying services that interfaced and integrated with other data processing services, as a set of web services organised in a 'grid' as part of the UK ASTRO-Grid contribution to the International Virtual Observatory.