How did it all start?

In March 2004 the International GNSS Service (IGS) celebrated its first decade of routine operations with a workshop and symposium in Berne, Switzerland. The accomplishments of the IGS over its first decade had been impressive, and had resulted in a stable service that routinely produced precise GPS orbits, satellite clocks and associated products. Perhaps more importantly, the IGS had created an atmosphere of constructive cooperation among the main GNSS analysis centres and network operators in the world, which accelerated progress in geodetic GNSS analysis in a significant way.

However, every silver cloud has a few dark spots in the middle. The structure of the IGS is such that almost all processing effort is carried by a handful of global Analysis Centres, on a voluntary basis only. The finite processing capacity of these centres directly constrains the analysis capacity of the IGS as a whole. Furthermore, the centralized nature of IGS in data centres, analysis centres, combination centres, sometimes makes it difficult or impossible to get access to important GNSS datasets in near real-time. These were the two main drivers behind the GPS Dancer project, which started around 2007 as a voluntary project of the International Association of Geodesy (IAG).

IGS processing capacity

By 2004 the processing capacity of the IGS was falling behind with the increasing demands of GNSS reference frame analysis:

  • In support of real-time GNSS analysis it would be desirable to reduce the product update rate from 6 hours (... IGS ultra-rapid) to 1 hour or less. This augments the workload for the analysis centres proportionally, by a factor 6 or more.
  • In support of combined orbit determination with Low Earth Orbiters (LEO) such as CHAMP, it would be necessary to analyze all GNSS data at sample intervals of 30 seconds, rather than 5 minutes. This augments the dataset - hence the processing workload of the analysis centres - by a factor 10.
  • In support of new GNSS constellations such as GLONASS, Beidou and Galileo, the number of satellites in the analysis would grow by a factor 5 or more, and the workload at the analysis centres would grow proportionally.
  • The latest generations of GNSS satellites have a third carrier frequency, which augments the GNSS observation dataset - and therefore the processing workload at the analysis centres - by another 50%.
  • In order to densify the global ITRF solution by a factor 10 - let alone 100 - the workload at the analysis centres would grow by the same factor.

Ideally, IGS would like to do all these things simultaneously, but this required an increase in processing capacity by a factor 3,000 or so. It would be unreasonable to expect this kind of effort from the existing analysis centres, and it would be unrealistic to expect that a few hundred new centres could be found to do all this extra work.

This means that the IGS remains the only IAG service that does not produce a complete ITRF realization for all permanent reference stations.

Data access

By 2004 there were an estimated 10,000 permanent GPS reference stations in the world, in regional densification networks, meteorological networks, search-and-rescue networks along the coast, and many more. Fewer than 3,500 of these stations published their data at short latency through regional or global data centres. From these public sites, only 350 stations were routinely processed by the IGS.

Among the datasets that were not available to IGS at short latency were important datasets from LEO receivers, which covered areas of Earth with little or no terrestrial stations (such as the Pacific Ocean, or the polar latitudes). It is understandable that operators of such satellites are reluctant to hand out raw observation data as soon as it is downloaded from their satellite: after investing millions in these LEO missions, the operators obviously have the right to extract most significant science results from the data, before releasing it to the public. In many cases, the main reason why the data is eventually published is to allow independent verification of science results produced by the LEO mission itself.

Most terrestrial receivers that do not publish their data also have valid motivations, for instance, protection of national interests, security of stations and users, or economic motivations. A country like China is perfectly entitled to not hand over the data from their GPS reference network to the international science community. Unfortunately, lack of data access limits the quality and relevance of the formal ITRF solution,

The solution: distributed GNSS analysis

If you want to run an analysis process that is a thousand times larger than what reasonably fits in a single computer, your only hope is to spread the workload over a thousand computers that somehow work together. This is the logic behind grid computing, which was already a well-established discipline by the year 2004.

If your input data cannot come to the analysis process, the only alternative is to move (part of) the process to the input data. Fortunately this becomes rather easy in a grid computing approach. On today's powerful internet, there is no reason why a cluster of a thousand computers should be geographically collocated at a single site. In fact, this would probably just be awkward.

Combining the two boundary conditions above, it appears that the problems of the IGS can (...only!) be solved by implementing a global GNSS network solution in the form of a grid computing process on the internet. We could argue that the network of global IGS analysis centres already performs a similar distributed solution process. However, the granularity of this network of analysis centres is too coarse. The data from hundreds of stations is processed by a single IGS analysis centre computer, while the input data must be published to the analysis centre if it wants to be part of the solution.

A refinement of the process granularity can allow many more computers to be involved in routine GNSS analysis. If the analysis is split in separate processing tasks per receiver, the input data no longer needs to be published, only some internal process information is exchanged with similar processes on-line, without requiring either the input data nor the output products to be published to the whole world. With one process per receiver, the granularity of the analysis becomes he same as that of the GNSS station network itself, and the computation process becomes scalable in the number of receivers. For every further receiver, we add a further process, which can run on an additional computer. Observations from LEO receivers can be processsed by their master telemetry download station, without any need to send the raw observation data to the IGS in near real-time. China can process its 2000 reference stations in private cloud computing accounts, and have them fully integrated in the ITRF without having to publish data to the IGS.

Whether we like it or not: distributed GNSS analysis at the level of individual receivers makes more sense than distributed analysis at the level of IGS analysis centres, both from a technical point of view (processing capacity), and from a political point of view (privacy).

Welcome to the GPS Dancer project

By the year 2006, a working group of the IAG with the baroque name "Combination and comparison of precise satellite orbits based on different geodetic techniques" also wanted to include LEO GNSS data in ITRF reference frame realizations, but would preferrably do so in combination with SLR, DORIS and altimetry data. This could contribute to reaching the GGOS objective of 1 mm consistency in reference frame geometry, and would hopefully remove some recognized inconsistencies between the ITRF realizations of the different geodetic techniques (notably, GPS and SLR). To this purpose the IAG Working Group pursued an integral reprocessing of all relevant space geodetic datasets, in contrast to the separate reprocessing efforts of the individual IAG services that were already taking place. 

As this analysis was clearly too large for any individual analysis centre, a project was initiated with the name Digger, which later became Dagger (Distributed Analysis for Global Geodetic Experimentation and Reprocessing). A fully functional prototype of a Dagger system, based on the Berkeley Open Infrastructure for Network Computing (BOINC), was up and running by the end of 2006. Unfortunately this prototype used proprietary software from ESA for the core parameter estimation element, and this software could not be freely distributed in the context of an IAG project. It merely demonstrated the feasibility of the Dagger concept.

What was needed, was independent parameter estimation software that could be freely distributed to thousands of computers in the world, and that was flexible enough to support a variety of planned reprocessing tasks. Such software did not exist, so that it would have to be developed first. Moreover, it had to be developed in a way that avoided IPR problems, which meant that none of the typical agencies (ESA, NASA, GFZ, CNES, ...) could really be expected to fund this project, not even if they wanted to. A voluntary project was initiated for the development of the parameter estimation core of the Dagger system, to be distributed under an open source license such as GPL or Apache.

The need for distributed analysis of the GNSS dataset, as discussed above, had already been recognized at that time. The voluntary IAG project therefore focussed on GPS as its primary geodetic dataset, with the objective of adding support for the other space geodetic techniques (notably SLR and VLBI) in a later stage. The project - or the software - did not really have a name at that time: it was seen as a part of the Dagger project.

During the early development stages an algorithm had to be worked out for efficient accumulation of distributed normal equation information. This algorithm was soon called "square dancing", becase it resembled the changing pair formations in traditional line dancing schemes. The individual processes that took part in the exchange scheme were referred to as dancers, and in the end the entire project was called IAG Dancer. The initial version of the software, which only processes GPS data, was called GPS Dancer.

Brief history of the Dancer project

The initial two years (2007 - 2008) of the project were merely spent on analysis and small feasibility studies, while various approaches were considered to actually implement the software. As the project had no budget, no suitable approach was found other than having volunteers who worked on the system in their own time. As strange as it sounds this works very well, as long as a non-zero number of volunteers can be found. The entire GPS analysis part of the software had been implemented and tested by late 2009 2010, and a prototype communication layer had been built around the JXTA protocol from SUN Microsystems by mid 2010. This initial system was for instance presented at the IGS Workshop in Newcastle, and at the AGU Fall assembly of 2010.

The years 2011 - 2012 were mainly spent on solving a variety of network communication issues. The Dancer process has extremely strict requirements in the area of "network volatility robustness", meaning that it must cope with the fact that any receiver process may go off-line at any moment during the computation process. This ultimately required a redesign of most of the network protection layer. Although this took a lot of time, the final result was a perfectly robust network logic that was no longer based on the original "buddy triangle" protection mechanism, but on a remarkably clever insight from Berkely University that we could make most of the difficult problems disappear by ignoring them completely.

By the time of the IGS Workshop 2012, the Dancer software was essentially ready for on-line testing and validation. The first permanently running Dancer processes were launched on a server in Germany (... the same server that hosts this website). Various minor technical issues were still pending, but emphasys has gradually shifted from purely technical development towards organisational issues. An intermediate Foundation appeared to be necessary between the Apache Foundation and the operational Dancer network, in order to host some crucial encryption keys of the run-time Dancer network. Furthermore, proper funding has been secured from ESA and the federal state of Hesse for launching an ITRF network solution in the cloud.

GPS Dancer has been a regular appearance on IGS Workshops, AGU Fall meetings in San Francisco and IAG meetings over the period 2010 - 2015. For details on algorithms, project organisation, published papers, and of course the software itself, please look around on this website. It acts as first source of information for any interested party, but also forms an historic log of the GPS Dancer project and the main source of documentation on the GPS Dancer analysis and software.

Project details

Ultimate Browsers SupportThe GPS Dancer project started in 2007 as a voluntary project of a working group of the International Association of Geodesy.

Read more ...

Square dance algorithm

Great Docs and SupportThe GPS Dancer system was named after its "square dance" exchange algorithm. Of course, it also wants to to make the GPS reference frame denser.


Here, there be pirates

The Dancer on-line network became immune against internet connection problems by leaving the US marines, and becoming a pirate.

Read more ...

Go to top