Package gavo :: Package utils :: Module fitstools
[frames] | no frames]

Module fitstools

source code

Some utility functions to deal with FITS files.

Note: pyfits is not thread-safe at least up to version 3.0.8. We therefore provide the fitsLock context manager here that you should use to protect places where you use pyfits in a core (or a similar spot).

Classes
  FITSError
  PlainHeaderManipulator
A class that allows header manipulation of fits files without having to touch the data.
  GzHeaderManipulator
is a class that allows header manipulation of fits files without having to touch the data even for gzipped files.
  WCSAxis
represents a single 1D WCS axis and allows easy metadata discovery.
  ESODescriptorsError
is raised when something goes wrong while parsing ESO descriptors.
Functions
 
fitsLock(*args, **kwds) source code
 
padCard(input, length=80)
pads input (a string) with blanks until len(result)%80=0
source code
 
readHeaderBytes(f, maxHeaderBlocks=80)
returns the bytes beloning to a FITS header starting at the current position within the file f.
source code
 
readPrimaryHeaderQuick(f, maxHeaderBlocks=80)
returns a pyfits header for the primary hdu of the opened file f.
source code
 
parseCards(aString)
returns a list of pyfits Cards parsed from aString.
source code
 
serializeHeader(hdr)
returns the FITS serialization of a FITS header hdr.
source code
 
replacePrimaryHeader(inputFile, newHeader, targetFile, bufSize=100000)
writes a FITS to targetFile having newHeader as the primary header, where the rest of the data is taken from inputFile.
source code
 
replacePrimaryHeaderInPlace(fitsName, newHeader)
replaces the primary header of fitsName with newHeader.
source code
 
sortHeaders(header, commentFilter=None, historyFilter=None, cardSequence=None)
returns a pyfits header with "real" cards first, then history, then comment cards.
source code
 
openGz(fitsName, tempDir=None)
returns the hdus for the gzipped fits fitsName.
source code
 
writeGz(hdus, fitsName, compressLevel=5, mode=436)
writes and gzips hdus into fitsName.
source code
 
openFits(fitsName)
returns the hdus for fName.
source code
 
HeaderManipulator(fName)
returns a header manipulator for a FITS file.
source code
 
getPrimarySize(fName)
returns x and y size a fits image.
source code
 
getAxisLengths(hdr)
returns a sequence of the lengths of the axes of a FITS image described by hdr.
source code
 
cutoutHeader(header, *cuts)
returns a array slices and a new pyfits header reflecting cuts.
source code
 
cutoutFITS(hdu, *cuts)
returns a cutout of hdu restricted to cuts.
source code
 
shrinkWCSHeader(oldHeader, factor)
returns a FITS header suitable for a shrunken version of the image described by oldHeader.
source code
 
iterFITSRows(hdr, f)
iterates over the rows of a FITS (primary) image.
source code
 
fixImageExtind(hdus, extInd)
returns the index of a compressed image HDU if it immediately follows extInd.
source code
 
iterScaledRows(inFile, factor=None, destSize=None, hdr=None, slow=False, extInd=0)
iterates over numpy arrays of pixel rows within the open FITS stream inFile scaled by it integer in factor.
source code
 
iterScaledBytes(inFileName, factor, extraCards={})
iterates over the bytes for a simple FITS file generated by scaling down the 2D image inFileName by factor.
source code
 
headerFromDict(d)
returns a primary header sporting the key/value pairs given in the dictionary d.
source code
 
getWCSAxis(header, axisIndex, forceSeparable=False)
returns a WCSAxis instance from an axis index and a FITS header.
source code
 
parseESODescriptors(hdr)
returns parsed ESO descriptors from a pyfits header hdr.
source code
Variables
  CARD_SIZE = 80
  END_CARD = 'END ...
  FITS_BLOCK_SIZE = 2880
  STANDARD_CARD_SEQUENCE = [('SIMPLE', True), ('BITPIX', True), ...
  NUM_CODE = {-64: '>f8', -32: '>f4', 8: 'uint8', 16: '>i2', 32:...
  __package__ = 'gavo.utils'
  ex = AttributeError("'module' object has no attribute 'showwar...
Function Details

fitsLock(*args, **kwds)

source code 
Decorators:
  • @contextmanager

padCard(input, length=80)

source code 

pads input (a string) with blanks until len(result)%80=0

The length keyword argument lets you adjust the "card size". Use this to pad headers with length=FITS_BLOCK_SIZE

>>> padCard("")
''
>>> len(padCard("end"))
80
>>> len(padCard("whacko"*20))
160
>>> len(padCard("junkodumnk"*17, 27))%27
0

readHeaderBytes(f, maxHeaderBlocks=80)

source code 

returns the bytes beloning to a FITS header starting at the current position within the file f.

If the header is not complete after reading maxHeaderBlocks blocks, a FITSError is raised.

readPrimaryHeaderQuick(f, maxHeaderBlocks=80)

source code 

returns a pyfits header for the primary hdu of the opened file f.

This is mostly code lifted from pyfits._File._readHDU. The way that class is made, it's hard to use it with stuff from a gzipped source, and that's why this function is here. It is used in the quick mode of fits grammars.

This function is adapted from pyfits.

parseCards(aString)

source code 

returns a list of pyfits Cards parsed from aString.

This will raise a ValueError if aString's length is not divisible by 80. It may also raise pyfits errors for malformed cards.

Empty (i.e., all-whitespace) cards are ignored. If an END card is encountered processing is aborted.

replacePrimaryHeader(inputFile, newHeader, targetFile, bufSize=100000)

source code 

writes a FITS to targetFile having newHeader as the primary header, where the rest of the data is taken from inputFile.

inputFile must be a file freshly opened for reading, targetFile one freshly opened for writing.

This function is (among other things) a workaround for pyfits' misfeature of unscaling scaled data in images when extending a header.

replacePrimaryHeaderInPlace(fitsName, newHeader)

source code 

replaces the primary header of fitsName with newHeader.

Doing this, it tries to minimize the amount of writing necessary; if fitsName has enough space for newHeader, just the header is written, and newHeader is extended if necessary. Only if newHeader is longer than the existing header is fitsName actually copied. We try to be safe in this case, only overwriting the old entry when the new data is safely on disk.

gzipped inputs used to be supported here, but they aren't any more.

sortHeaders(header, commentFilter=None, historyFilter=None, cardSequence=None)

source code 

returns a pyfits header with "real" cards first, then history, then comment cards.

Blanks in the input are discarded, and one blank each is added in between the sections of real cards, history and comments.

Header can be an iterable yielding Cards or a pyfits header.

Duplicate history or comment entries will be swallowed.

cardSequence, if present, is a sequence of (item, mandatory) pairs. There item can be a card name, in which case the corresponding card will be inserted at this point in the sequence. If mandatory is True, a missing card is an error. Keywords already in fitstools.STANDARD_CARD_SEQUENCE are ignored.

Item can also a pyfits.Card instance; it will be put into the header as-is.

As a shortcut, a sequence item may be something else then a tuple; it will then be combined with a False to make one.

These days, if you think you need this, have a look at fitstricks.makeHeaderFromTemplate first.

openGz(fitsName, tempDir=None)

source code 

returns the hdus for the gzipped fits fitsName.

Scrap that as soon as we have gzipped fits support (i.e. newer pyfits) in debian.

writeGz(hdus, fitsName, compressLevel=5, mode=436)

source code 

writes and gzips hdus into fitsName. As a side effect, hdus will be closed.

Appearently, not even recent versions of pyfits support writing of zipped files (which is a bit tricky, admittedly). So, we'll probably have to live with this kludge for a while.

openFits(fitsName)

source code 

returns the hdus for fName.

(gzip detection is tacky, and we should look at the magic).

This is simulating legacy pyfits behaviour in that it will use memmap I/O if it thinks that works. You'll probably want to use astrpy directly these days if this gives you a headache.

HeaderManipulator(fName)

source code 

returns a header manipulator for a FITS file.

(it's supposed to look like a class, hence the uppercase name) It should automatically handle gzipped files.

cutoutHeader(header, *cuts)

source code 

returns a array slices and a new pyfits header reflecting cuts.

(see cutoutFits for what cuts is).

cutoutFITS(hdu, *cuts)

source code 

returns a cutout of hdu restricted to cuts.

hdu is a primary FITS hdu. cuts is a list of cut specs, each of which is a triple (axis, lower, upper). axis is between 1 and naxis, lower and upper a 1-based pixel coordinates of the limits, and "border" pixels are included. Specifications outside of the image are legal and will be cropped back. Open limits are supported via a specification of None.

If an axis would vanish (i.e. length 0 or less), the function fudges things such that the axis gets a length of 1.

axis is counted here in the FORTRAN/FITS sense, *not* in the C sense, i.e., axis=1 cuts along NAXIS1, which is the *last* index in a numpy array.

WCS CRPIXes in hdu's header will be updated. Axes and specified will not be touched. It is an error to specifiy cuts for an axis twice (behaviour is undefined).

Note that this will lose all extensions the orginal FITS file might have had.

shrinkWCSHeader(oldHeader, factor)

source code 

returns a FITS header suitable for a shrunken version of the image described by oldHeader.

This only works for 2d images, scale must be an integer>1. The function assumes no "fractional" pixels are handled, i.e, remainders in divisions with factors are discarded. It is thus a companion for iterScaledRows.

Note that oldHeader must be an actual pyfits header instance; a dictionary will not do.

This is a pretty straight port of wcstools's imutil.c#ShrinkFITSHeader, except we clear BZERO and BSCALE and set BITPIX to -32 (float array) under the assumption that in the returned image, 32-bit floats are used.

iterFITSRows(hdr, f)

source code 

iterates over the rows of a FITS (primary) image.

You pass in a FITS header and a file positioned to the start of the image data.

What's returned are 1d numpy arrays of the datatype implied by bitpix. The function understands only very basic FITSes (BSCALE and BZERO are known, though, and lead to floats arrays).

We do this ourselves since pyfits may pull in the whole thing or at least mmaps it; both are not attractive when I want to stream-process large images.

fixImageExtind(hdus, extInd)

source code 

returns the index of a compressed image HDU if it immediately follows extInd.

If hdus[extInd] is a CompImageHDU itself, this will return extInd unchanged.

iterScaledRows(inFile, factor=None, destSize=None, hdr=None, slow=False, extInd=0)

source code 

iterates over numpy arrays of pixel rows within the open FITS stream inFile scaled by it integer in factor.

The arrays are always float32, regardless of the input. When the image size is not a multiple of factor, border pixels are discarded.

A FITS header for this data can be obtained using shrinkWCSHeader.

You can pass in either a factor the file is to be scaled down, or an approximate size desired. If both are given, factor takes precedence, if none is given, it's an error.

If you pass in hdr, it will not be read from inFile; the function then expects the file pointer to point to the start of the first data block. Use this if you've already read the header of a non-seekable FITS.

extInd lets you select a different extension. extInd=0 means the first image HDU, which may be in extension 1 for compressed images.

iterScaledRows will try to use a hand-hacked interface guaranteed to stream. This only works for plain, 2D-FITSes from real files. iterScaledRows normally notices when it should fall back to pyfits code, but if it doesn't you can pass slow=True.

iterScaledBytes(inFileName, factor, extraCards={})

source code 

iterates over the bytes for a simple FITS file generated by scaling down the 2D image inFileName by factor.

factor must be an integer between 1 and something reasonable.

This function acquires the FITS lock itself.

headerFromDict(d)

source code 

returns a primary header sporting the key/value pairs given in the dictionary d.

In all likelihood, this header will *not* work properly as a primary header because, e.g., there are certain rules on the sequence of header items. fitstricks.copyFields can make a valid header out of what's returned here.

keys mapped to None are skipped, i.e., you have to do nullvalue handling yourself.

getWCSAxis(header, axisIndex, forceSeparable=False)

source code 

returns a WCSAxis instance from an axis index and a FITS header.

If the axis is mentioned in a transformation matrix (CD or PC), a ``ValueError`` is raised (use ``forceSeparable`` to override).

The ``axisIndex`` is 1-based; to get a transform for the axis described by CTYPE1, pass 1 here.

The object returned has methods like ``pixToPhys``, ``physToPix`` (and their ``pix0`` brethren), and ``getLimits``.

Note that at this point WCSAxis only supports linear transforms (it's a DaCHS-specific implementation). We'll extend it on request.

Decorators:
  • @codetricks.document

parseESODescriptors(hdr)

source code 

returns parsed ESO descriptors from a pyfits header hdr.

ESO descriptors are data columns stuck into FITS history lines. They were produced by MIDAS. This implementation was made without actual documentation, is largely based on conjecture, and is certainly incomplete.

What's returned is a dictionary mapping column keywords to lists of column values.


Variables Details

END_CARD

Value:
'END                                                                  \
           '

STANDARD_CARD_SEQUENCE

Value:
[('SIMPLE', True),
 ('BITPIX', True),
 ('NAXIS', True),
 ('NAXIS1', False),
 ('NAXIS2', False),
 ('NAXIS3', False),
 ('NAXIS4', False),
 ('EXTEND', False),
...

NUM_CODE

Value:
{-64: '>f8', -32: '>f4', 8: 'uint8', 16: '>i2', 32: '>i4', 64: '>i8'}

ex

Value:
AttributeError("'module' object has no attribute 'showwarning'",)