idlastro / FITS I/O: MRDFITS

[Source code]

NAME
MRDFITS
PURPOSE
Read all standard FITS data types into arrays or structures.
EXPLANATION
Further information on MRDFITS is available at
http://idlastro.gsfc.nasa.gov/mrdfits.html 
CALLING SEQUENCE
Result = MRDFITS( Filename/FileUnit,[Exten_no/Exten_name, Header],
                 /FPACK, /NO_FPACK, /FSCALE , /DSCALE , /UNSIGNED,
                 ALIAS=strarr[2,n], /USE_COLNUM,
                 /NO_TDIM, ROWS = [a,b,...], $
                 /POINTER_VAR, /FIXED_VAR, EXTNUM= 
                 RANGE=[a,b], COLUMNS=[a,b,...]), ERROR_ACTION=x,
                 COMPRESS=comp_prog, STATUS=status, /VERSION, 
                 /EMPTYSTRING )
INPUTS
Filename = String containing the name of the file to be read or
           file number of an open unit.  If an empty string is supplied,
           then user will be prompted for the file name.    The user
           will also be prompted if a wild card is given in the file
           name, and there is more than one file name match.
           If the file name ends in .gz or .fz (or .Z on Unix systems)
           the file will be dynamically decompressed.
                              or
FiluUnit = An integer file unit which has already been
           opened for input.  Data will be read from this
           unit and the unit will be left pointing immediately
           after the HDU that is read.  Thus to read a compressed
           file with many HDU's a user might do something like:
                lun=fxposit(filename, 3)  ; Skip the first three HDU's
                repeat begin
                    thisHDU = mrdfits(lun, 0, hdr, status=status)
                    ... process the HDU ...
                endrep until status lt 0
Exten_no= Extension number to be read, 0 for primary array.
           Assumed 0 if not specified.
           If a unit rather than a filename
           is specified in the first argument, this is
           the number of HDU's to skip from the current position.
Exten_name - Name of the extension to read (as stored in the EXTNAME
           keyword).   This is a slightly slower method then specifying
           the extension number.
OUTPUTS
Result = FITS data array or structure constructed from
         the designated extension.  The format of result depends
         upon the type of FITS data read.
       Non-group primary array or IMAGE extension:
         A simple multidimensional array is returned with the
         dimensions given in the NAXISn keywords.
       Grouped image data with PCOUNT=0.
         As above but with GCOUNT treated as NAXIS(n+1).
       Grouped image data with PCOUNT>0.
         The data is returned as an array of structures.  Each
         structure has two elements.  The first is a one-dimensional
         array of the group parameters, the second is a multidimensional
         array as given by the NAXIS2-n keywords.
       ASCII and BINARY tables.
         The data is returned as a structure with one column for
         each field in the table.  The names of the columns are
         normally taken from the TTYPE keywords (but see USE_COLNUM).
         Bit field columns
         are stored in byte arrays of the minimum necessary
         length.  Spaces and invalid characters are replaced by 
         underscores, and other invalid tag names are converted using
         the IDL_VALIDNAME(/CONVERT_ALL) function.
         Columns specified as variable length columns are stored
         with a dimension equal to the largest actual dimension
         used.  Extra values in rows are filled with 0's or blanks.
         If the size of the variable length column is not
         a constant, then an additional column is created giving the 
         size used in the current row.  This additional column will 
         have a tag name of the form L#_"colname" where # is the column
         number and colname is the column name of the variable length
         column.   If the length of each element of a variable length 
         column is 0 then the column is deleted.
OPTIONAL OUTPUT
Header = String array containing the header from the FITS extension.
OPTIONAL INPUT KEYWORDS
ALIAS    The keyword allows the user to specify the column names
         to be created when reading FITS data.  The value of
         this keyword should be a 2xn string array.  The first
         value of each pair of strings should be the desired
         tag name for the IDL column.  The second should be
         the FITS TTYPE value.  Note that there are restrictions
         on valid tag names.  The order of the ALIAS keyword
         is compatible with MWRFITS.
COLUMNS - This keyword allows the user to specify that only a
         subset of columns is to be returned.  The columns
         may be specified either as number 1,... n or by
         name or some combination of these two.
         If /USE_COLNUM is specified names should be C1,...Cn.
         The use of this keyword will not save time or internal
         memory since the extraction of specified columns
         is done after all columns have been retrieved from the
         FITS file.      Structure columns are returned in the order
         supplied in this keyword.
COMPRESS - This keyword allows the user to specify a
         decompression program to use to decompress a file that
         will not be automatically recognized based upon
         the file name.
/DSCALE - As with FSCALE except that the resulting data is
         stored in doubles.
/EMPTYSTRING - There was a bug in memory management for IDL versions 
          prior to V8.0, causing a memory leak when reading
          empty strings in a FITS table.   Setting /EMPTYSTRING will
          avoid this problem by first reading strings into bytes and
          then converting.   However, there is a performance penalty.                 
ERROR_ACTION - Set the on_error action to this value (defaults
         to 2).
/FIXED_VAR- Translate variable length columns into fixed length columns
         and provide a length column for truly varying columns.
         This was the only behavior prior to V2.5 for MRDFITS and remains
         the default (see /POINTER_VAR)
/FPACK - If set, then assume the FITS file uses FPACK compression 
         (http://heasarc.gsfc.nasa.gov/fitsio/fpack/).     To read
         an FPACK compressed file, either this must be set or the 
         file name must end in ".fz"
/NO_FPACK - If present, then MRDFITS will not uncompress an extension
         compressed with FPACK (i.e with a .fz extension), but will 
         just read the compressed binary stream. 
/FSCALE - If present and non-zero then scale data to float
         numbers for arrays and columns which have either
         non-zero offset or non-unity scale.
         If scaling parameters are applied, then the corresponding
         FITS scaling keywords will be modified.
NO_TDIM  - Disable processing of TDIM keywords.  If NO_TDIM
         is specified MRDFITS will ignore TDIM keywords in
         binary tables.
/POINTER_VAR- Use pointer arrays for variable length columns.
         The pointer tag must be dereferenced in the output structure
         to access the variable length column.   Prior to IDL V8.0, the user 
         was responsible for memory management when deleting or reassigning 
         the structure (e.g. using HEAP_FREE), but memory management of pointer
         arrays is now automatic.         
RANGE  - A scalar or two element vector giving the start
         and end rows to be retrieved.  For ASCII and BINARY
         tables this specifies the row number.  For GROUPed data
         this will specify the groups.  For array images, this
         refers to the last non-unity index in the array.  E.g.,
         for a 3 D image with NAXIS* values = [100,100,1], the
         range may be specified as 0:99, since the last axis
         is suppressed.  Note that the range uses IDL indexing
         So that the first row is row 0.
         If only a single value, x, is given in the range,
         the range is assumed to be [0,x-1].
ROWS -  A scalar or vector specifying a  specific row or rows to read 
        (first row is 0).   For example to read rows 0,
        12 and 23 only, set ROWS=[0,12,23].   Valid for images, ASCII 
        and binary tables, but not GROUPed data.   For images
        the row numbers refer to the last non-unity index in the array.
        Note that the use of the ROWS will not improve the speed of
        MRDFITS since the entire table will be read in, and then subset
        to the specified rows.     Cannot be used at the same time as 
        the RANGE keyword
/SILENT - Suppress informative messages.
STRUCTYP - The structyp keyword specifies the name to be used
         for the structure defined when reading ASCII or binary
         tables.  Generally users will not be able to conveniently
         combine data from multiple files unless the STRUCTYP
         parameter is specified.  An error will occur if the
         user specifies the same value for the STRUCTYP keyword
         in calls to MRDFITS in the same IDL session for extensions
         which have different structures.
/UNSIGNED - For integer data with appropriate zero points and scales
         read the data into unsigned integer arrays.
/USE_COLNUM - When creating column names for binary and ASCII tables
         MRDFITS attempts to use the appropriate TTYPE keyword
         values.  If USE_COLNUM is specified and non-zero then
         column names will be generated as 'C1, C2, ... 'Cn'
         for the number of columns in the table.
/VERSION Print the current version number
OPTIONAL OUTPUT KEYWORDS
EXTNUM - the number of the extension actually read.   Useful if the
          user specified the extension by name.
OUTALIAS - This is a 2xn string array where the first column gives the
         actual structure tagname, and the second gives the
         corresponding FITS keyword name (e.g. in the TTYPE keyword).   
         This array can be passed directly to
         the alias keyword of MWRFITS to recreate the file originally
         read by MRDFITS.
STATUS - A integer status indicating success or failure of
         the request.  A status of >=0 indicates a successful read.
         Currently
             0 -> successful completion
            -1 -> error
            -2 -> end of file
EXAMPLES
(1) Read a FITS primary array:
        a = mrdfits('TEST.FITS')    or
        a = mrdfits('TEST.FITS', 0, header)
The second example also retrieves header information.
(2) Read rows 10-100 of the second extension of a FITS file.
        a = mrdfits('TEST.FITS', 2, header, range=[10,100])
(3) Read a table and ask that any scalings be applied and the
scaled data be converted to doubles.  Use simple column names,
suppress outputs.
        a = mrdfits('TEST.FITS', 1, /dscale, /use_colnum, /silent)
(4) Read rows 3, 34 and 52 of a binary table and request that 
    variable length columns be stored as a pointer variable in the 
    output structure
       a = mrdfits('TEST.FITS',1,rows=[3,34,52],/POINTER)
RESTRICTIONS
(1)     Cannot handle data in non-standard FITS formats.
(2)     Doesn't do anything with BLANK or NULL values or
        NaN's.  They are just read in.  They may be scaled
        if scaling is applied.
(3)     Does not automatically detect a FPACK compressed file.  Either
        the file name must end in .fz, or the /FPACK keyword must
        be set
NOTES
This multiple format FITS reader is designed to provide a
single, simple interface to reading all common types of FITS data.
MRDFITS DOES NOT scale data by default.  The FSCALE or DSCALE
parameters must be used.
Null values in an FITS ASCII table are converted to NaN (floating data),
or -2147483647L (longwords) or '...' (strings).   
PROCEDURES USED
The following procedures are contained in the main MRDFITS program.
    MRD_IMAGE           -- Generate array/structure for images.
    MRD_READ_IMAGE      -- Read image data.
    MRD_ASCII           -- Generate structure for ASCII tables.
    MRD_READ_ASCII      -- Read an ASCII table.
    MRD_TABLE           -- Generate structure for Binary tables.
    MRD_READ_TABLE      -- Read binary table info.
    MRD_READ_HEAP       -- Read variable length record info.
    MRD_SCALE           -- Apply scaling to data.
    MRD_COLUMNS         -- Extract columns.
 Other ASTRON Library routines used
    FXPAR(), FXADDPAR, FXPOSIT, FXMOVE(), MATCH, MRD_STRUCT(), MRD_SKIP 
MODIfICATION HISTORY
V1.0 November 9, 1994 ----  Initial release.
   Creator: Thomas A. McGlynn
V1.1 January 20, 1995 T.A. McGlynn
   Fixed bug in variable length records.
   Added TDIM support -- new routine mrd_tdim in MRD_TABLE.
V1.2
   Added support for dynamic decompression of files.
   Fixed further bugs in variable length record handling.
V1.2a
   Added NO_TDIM keyword to turn off TDIM processing for
   those who don't want it.
   Bug fixes: Handle one row tables correctly, use BZERO rather than
        BOFFSET.     Fix error in scaling of images.  
V1.2b 
   Changed MRD_HREAD to handle null characters in headers.
V2.0 April 1, 1996
   -Handles FITS tables with an arbitrary number of columns.
   -Substantial changes to MRD_STRUCT to allow the use of
   substructures when more than 127 columns are desired.
   -All references to table columns are now made through the
   functions MRD_GETC and MRD_PUTC.  See description above.
   -Use of SILENT will now eliminate compilation messages for
   temporary functions.
   -Bugs in handling of variable length columns with either
   a single row in the table or a maximum of a single element
   in the column fixed.
   -Added support for DCOMPLEX numbers in binary tables (M formats) for
   IDL versions above 4.0.  
   -Created regression test procedure to check in new versions.
   -Added error_action parameter to allow user to specify
   on_error action.  This should allow better interaction with
   new CHECK facility.  ON_ERROR statements deleted from
   most called routines.
   - Modified MRDFITS to read in headers containing null characters
   with a warning message printed.
V2.0a April 16, 1996
   - Added IS_IEEE_BIG() checks (and routine) so that we don't
   worry about IEEE to host conversions if the machine's native
   format is IEEE Big-endian.
V2.1 August 24, 1996
   - Use resolve_routine for dynamically defined functions
   for versions > 4.0
   - Fix some processing in random groups format.
   - Handle cases where the data segment is--legally--null.
   In this case MRDFITS returns a scalar 0.
   - Fix bugs with the values for BSCALE and BZERO (and PSCAL and
   PZERO) parameters set by MRDFITS.
V2.1a April 24, 1997  Handle binary tables with zero length columns
V2.1b May 13,1997 Remove whitespace from replicate structure definition
V2.1c May 28,1997 Less strict parsing of XTENSION keyword
V2.1d June 16, 1997 Fixed problem for >32767 entries introduced 24-Apr
V2.1e Aug 12, 1997 Fixed problem handling double complex arrays
V2.1f Oct 22, 1997 IDL reserved words can't be structure tag names
V2.1g Nov 24, 1997 Handle XTENSION keywords with extra blanks.
V2.1h Jul 26, 1998 More flexible parsing of TFORM characters
V2.2 Dec 14, 1998 Allow fields with longer names for
                 later versions of IDL.
                 Fix handling of arrays in scaling routines.
                 Allow >128 fields in structures for IDL >4.0
                 Use more efficient structure copying for
                 IDL>5.0
V2.2b June 17, 1999 Fix bug in handling case where
                    all variable length columns are deleted
                    because they are empty.
V2.3 March 7, 2000 Allow user to supply file handle rather
                   than file name.
                   Added status field.
                   Now needs FXMOVE routine
V2.3b April 4, 2000
                   Added compress option (from D. Palmer)
V2.4  July 4, 2000 Added STATUS=-1 for "File access error" (Zarro/GSFC)
V2.4a May 2, 2001  Trim binary format string   (W. Landsman)
V2.5 December 5, 2001 Add unsigned, alias, 64 bit integers. version, $
                    /pointer_val, /fixed_var.
V2.5a Fix problem when both the first and the last character
     in a TTYPEnn value are invalid structure tag characters
V2.6 February 15, 2002 Fix error in handling unsigned numbers, $
                    and 64 bit unsigneds. (Thanks to Stephane Beland)
V2.6a September 2, 2002 Fix possible conflicting data structure for
                   variable length arrays (W. Landsman)
V2.7 July, 2003  Added Rows keyword (W. Landsman)
V2.7a September  2003 Convert dimensions to long64 to handle huge files
V2.8 October 2003 Use IDL_VALIDNAME() function to ensure valid tag names
                  Removed OLD_STRUCT and TEMPDIR keywords W. Landsman
V2.9 February 2004 Added internal MRD_FXPAR procedure for faster
              processing of binary table headers E. Sheldon
V2.9a March 2004 Restore ability to read empty binary table W. Landsman
      Swallow binary tables with more columns than given in TFIELDS
V2.9b Fix to ensure order of TFORMn doesn't matter
V2.9c Check if extra degenerate NAXISn keyword are present W.L. Oct 2004 
V2.9d Propagate /SILENT to MRD_HREAD, more LONG64 casting W. L. Dec 2004
V2.9e Add typarr[good] to fix a problem reading zero-length columns
      A.Csillaghy, csillag@ssl.berkeley.edu (RHESSI)
V2.9f Fix problem with string variable binary tables, possible math 
      overflow on non-IEEE machines  WL Feb. 2005 
V2.9g Fix problem when setting /USE_COLNUM   WL Feb. 2005
V2.10 Use faster keywords to BYTEORDER  WL May 2006
V2.11  Add ON_IOERROR, CATCH, and STATUS keyword to MRD_READ_IMAGE to
      trap EOF in compressed files DZ  Also fix handling of unsigned 
      images when BSCALE not present  K Chu/WL   June 2006 
V2.12 Allow extension to be specified by name, added EXTNUM keyword
              WL    December 2006
V2.12a Convert ASCII table column to DOUBLE if single precision is
          insufficient  
V2.12b Fixed problem when both /fscale and /unsigned are set 
           C. Markwardt    Aug 2007
V2.13  Use SWAP_ENDIAN_INPLACE instead of IEEE_TO_HOST and IS_IEEE_BIG
         W. Landsman Nov 2007
V2.13a One element vector allowed for file name W.L. Dec 2007
V2.13b More informative error message when EOF found W.L. Jun 2008
V2.14  Use vector form of VALID_NUM(), added OUTALIAS keyword
                                W.L. Aug 2008
V2.15  Use new FXPOSIT which uses on-the-fly byteswapping W.L. Mar 2009
V2.15a Small efficiency updates to MRD_SCALE W.L. Apr 2009
V2.15b Fixed typo introduced Apr 2009
V2.15c Fix bug introduced Mar 2009  when file unit used W.L. July 2009
V2.16  Handle FPACK compressed files    W. L. July 2009
V2.17  Use compile_opt hidden on all routines except mrdfits.pro W.L. July 2009
V2.18  Added /EMPTYSTRING keyword W. Landsman August 2009
V2.18a Fix Columns keyword output, A. Kimball/ W. Landsman Feb 2010
V2.18b Fix bug with /EMPTYSTRING and multidimensional strings
                      S. Baldridge/W.L. August 2010
V2.18c Fix unsigned bug caused by compile_opt idl2 WL  Nov 2010
V2.19  Use V6.0 operators WL Nov 2010
V2.19a Fix complex data conversion in variable length tables WL Dec 2010 
V2.19b Fix bug with /FSCALE introduced Nov 2010 WL Jan 2011
V2.19c Fix bug with ROWS keyword introduced Nov 2010 WL Mar 2011
V2.20  Convert Nulls in ASCII tables, better check of duplicate keywords
                                     WL May 2011
V2.20a Better error checking for FPACK files  WL October 2012
V2.20b Fix bug in MRD_SCALE introduced Nov 2010 (Sigh) WL Feb 2013
V2.21  Create unique structure tags when FITS column names differ 
       only in having a different case   R. McMahon/WL   March 2013
V2.22  Handle 64 bit variable length binary tables WL   April 2014
V2.23  Use 64 bit for  very large files  WL  April 2014
V2.24  Binary table is allowed to have zero columns  WL  September 2018