8. And now all-VO

The nice thing about standard services: Handle one, and you get them all. So, let’s add a query to the Registry and run our query all over the VO –

for svc in registry.search(servicetype="image", waveband="optical"):
  try:
    search_one_service(svc.accessurl, image_sender)
  except Exception:
    import traceback; traceback.print_exc()

[See globalsiap.py]

The Registry that’s being used here is a big directory of all the services that are in the VO. If you have tried the problem above and tried WIRR: it’s the same underlying data.

The Registry interface of PyVO is a bit simplistic. For a few basic tasks, it works ok, though. You will usually want to avoid doing “blind” all-VO queries, as they will take a long, and you’ll be hitting a lot of services that will return data you can’t use and just have to week out later. For cone searches, just don’t ever do it, since there are more than 104 of them

We will later see how to directly use RegTAP, the powerful and flexible Registry query technology; we recommend you use that for all but the most trivial discovery tasks.

Having said that, you can also search for catalog, spectrum, line, and database services, and waveband takes quite a few other terms (see docs for which these are). You can also pass keyword (for queries against titles and descriptions), and datamodel (which we’ll look at later) as further keyword arguments.

The exception catcher is there since not all services claiming to be standards-compliant actually are. It doesn’t hurt to complain to the service operators if a service you’re interested in behaves weirdly – sometimes the operators haven’t noticed it’s broken or just broke.

To find out who to complain to, you can again use the Registry. In the most common case, you would use WIRR with an access URL constraint. In the query result, you should see a mail address when clicking on the person icon.

You will probably also see lots of warnings from astropy’s VOTable parser. This is partly because astropy is overly paranoid, rejecting UCDs actually required by the SIAP standard, partly because operators botch things. Interoperability isn’t always easy. I’d say at this point it’s too early to complain to operators about your average VOTable warning, which is why we’ll later shut them off.

If a service hangs, you can interrupt it by hitting Control-C. In production code, you can use python’s socket.settimeout is your friend to fail broken services after a while and just go on.

Rule: In multi-service queries, expect at least one service to be broken. Write your scripts to cope.

Problems

(1)

Get the globalsiap.py script from the attachment and change it so it skips 90% of the services discovered randomly (use random.random()). Also, remove the constraint on the date (few services will have stuff this old) and change the position to something you’re interested in or expect to have pretty pictures (M1’s or M51’s are always good candiates). Run the thing and see what you find.

Files


Markus Demleitner, Hendrik Heinl