FITS image data format

[GeorgeCarey]

Are there any experts on FITS images out there?

 

I have a friend who is a whizz at programming and have coaxed him into developing software that can take a few FITS images and search for new objects.

In other words, automated nova/supernova detection.

 

He has had some success in reading a FITS image, despite the incomprehensible information about the data structure.

 

For some strange reason, we are getting the right hand 280 columns appearing at the left - despite reading the data sequentially from the start.

 

The image is 3326 x 2504 pixels.

 

Very stretched image below:

 

Any help would be much appreciated!

[GeorgeCarey]

Great progress. We are now able to extract adu values from the fits file that agree with Maxim.

The hardest part is done!

 

[LaCebra]

Hi George,

 

Several years ago, I processed "fits" images in an elementary way (autostrech).
For that I used a library of functions in C#, NASA origin (?)
There was indeed a "bug" in their source code. After correction everything worked well.

If necessary, I could look for these elements.

Good luck with that.
La Cebra

[Waldemar]

Hi George,

 

Several years ago, I processed "fits" images in an elementary way (autostrech).

For that I used a library of functions in C#, NASA origin (?)

There was indeed a "bug" in their source code. After correction everything worked well.

If necessary, I could look for these elements.

Good luck with that.

La Cebra

 

Hi La Cebra,

 

You mean the 'Fits Liberator' programm from the NASA?

Can I ask what you corrected in their source code?

 

Best regards,

Waldemar

[GeorgeCarey]

Thanks  La Cebra,

My friend programs in Delphi and it looks like there are no FITS libraries available.

He is having fun working it all out from scratch!

[LaCebra]

@George 

 

You can see that...

 

https://fits.gsfc.nasa.gov/fits_libraries.html#delafits
 

 

DeLaFits

DeLaFits (free software, MIT License) is published on GitHub. DeLaFits is native for Delphi and Lazarus and is not a wrapper around another library.

Features: building, reading, editing, and rendering of a FITS primary array image.

DeLaFits supports only Single Image (one header and one data unit).

[LaCebra]

@Waldemar

 

No, it's CSharpFITS - a complete C# library to FITS (but with a little bug  )

 

 

https://fits.gsfc.nasa.gov/fits_libraries.html#CSharpFITS

DownLoad :http://voi.iucaa.in/voi/CSharpFITS.html

[GeorgeCarey]

LaCebra - that looks very interesting! I will pass that along to Lutgen. (the programmer).

 

We have a small problem reading FITS from another friend of mine.

In his header there is a keyword 'BIAS' which does not seem normal. It is not to do with bias calibration frames.

It does not seem to be a standard keyword?

In his reference image he has BZERO=33753.0 and in other images it is the usual 32768.0  (2^15)

Also his reference image has BSCALE=0.9699392681661427    and in others it is the usual 1.0000

Very odd.

[LaCebra]

Hello George,

 

1 - I have a definite advantage with C# - CSharpFITS
I was able to extract the functions I needed, then decode line by line what needed to be done.
And then I copied/reprogrammed as needed (store my Bias, Darks, Flats, Lights images in SQL-Server).
I mean, I forgot everything   

 

It is frankly more difficult in Delphi where sources are scarcer.

 

 

2 - Can you publish one of these images in the cloud, describe the shooting (software, camera, etc.), and finally make a synthesis of your current analyses.
I'm not sure I have an immediate answer, however....

 

 

3 - Astronomical photography is of the same nature as Lucky Luke : A long and lonely journey

 

 

La Cebra

 

(Translated with www.DeepL.com/Translator)

[GeorgeCarey]

Good progress made. I decided to hack into the FITS files myself - the hard way!

I have no way of reading Hex values from a file so found a hex reader on the web, and a hex to decimal converter.

I can now read the 16 bit FITS files and get exactly the same adu values out as Maxim does.

I then wrote a spreadsheet that would do the task - success.

Lutgen found a bug in his code late last night - an overflow was occurring, so I think we are back on track.

 

Edited August 24, 2019 by GeorgeCarey

[GeorgeCarey]

And the spreadsheet - dealing with the peculiar values of BZERO and BSCALE:

 

[nakbrooks]

Catching up on this thread very late but if your friend is familiar with Python there are very extensive tools in the Astropy package and affiliated packages that would do pretty much everything you’d want. I’m sliwly building a web application, mostly using Astropy packages. This is primarily to support Photometry work.

[nakbrooks]

Astropy does require a good working knowledge of Python programming (under either Windows, IOS or Linux). It isn’t a shrink-wrapped package - it is a library of astronomy-related Python modules.

 

Core Astropy is documented here: http://docs.astropy.org/en/stable/

 

Astropy-affiliated packages are numerous. The most relevant include:

 

ccdproc (FITS image processing) https://ccdproc.readthedocs.io/en/latest/

 

photutils (source detection and photometry) https://photutils.readthedocs.io/en/stable/

 

Astroquery (automated querying of Vizier, Simbad and dozens of other data sources): https://astroquery.readthedocs.io/en/latest/

 

astroplan (scheduling observations based on observability and other constraints) https://astroplan.readthedocs.io/en/latest/

 

Python isn’t a difficult language to learn (easier than C# in my view). I’m 67 years old and my previous programming experience has been Excel macros and VBA but I’ve picked it up well enough in just 9 months to write a full data reduction pipeline.

 

Best of all it’s totally free.

 

I’m keen to work with other amateur Python programmers as I’m looking to open source what I’m doing.

 

Nigel

[nakbrooks]

Also, if you’ve not already located them:

 

The formal FITS file standard:

https://fits.gsfc.nasa.gov/standard40/fits_standard40aa-le.pdf

 

The SBIG FITS extensions (an industry standard followed by Maxim and most camera drivers):

http://www.company7.com/library/sbig/pdffiles/SBFITSEXT_1r0.pdf

[GeorgeCarey]

Thanks, I have seen most of those links. We can read the files fine now and convert to bmp.

I have been looking at astrometry. From the FITS header we have the image centre coordinates and plate scale plus rotation.

I have got a spreadsheet working to find the coordinates of an object so that the relevant area can be downloaded from PanSTARRS or Simbad.

Far from the centre of the image errors creep in. The process of converting pixel coordinates to real world is horrendously complicated.

My system uses TAN - gnomonic projection (there are about 20 different projections!)

I am not sure I am up to sorting that out.

[nakbrooks]

Yes that’s why I use Astropy. It handles all that stuff.

[ChristerS]

Hi George,

 

I have a complete C# project reading the Nasa CFITSIO lib https://heasarc.gsfc.nasa.gov/fitsio/and convert it to different other formats, I can share some code. Pixinsight also uses this lib. Im developing a database driven astro solution to handle a lot of images searcheble and hadle large stacks ready Q1 2020.

 

Cheers,

//Christer

http://astropix.se/
 

[GeorgeCarey]

Thanks. I posed a question on Astronomy StackExchange and everyone says 'Python'! It looks like I will have to learn some Python programming.

There is a Python interface to the Nasa CFITSIO lib so I will have a rummage through that.

[nakbrooks]

Yes it is far and away the most popular language for professional astronomers and is well supported by the Astropy and wider Python community.

[nakbrooks]

If you do get into using Python then make sure you use the Anaconda Python framework - it is free and includes all the scientific/astronomy Python packages you’re likely to need so makes things easier.

 

For program writing I strongly recommend Microsoft Visual Studio Code (NOT Visual Studio itself which is much more complex). VS Code is also free and has a Python extension which does syntax checking etc.

[nakbrooks]

George, was your immediate objective to extract star locations from fits images?

 

If so then you may want to look at astrometry.net. This is a free platesolving service that, as a byproduct, will give the xy coordinates of all stars in the image.

 

There is an api which can be accessed using Python but you can also do it interactively here: http://nova.astrometry.net/

 

Note that the distortion coefficients generated (which seek to adjust for optical distortions, particularly away from the image centre) use the professional system, not the one used by most amateur tools such as Maxim and PinPoint, so if you use astrometry.net platesolve solutions for subsequent processing you’ll either need to use Astropy or platesolve again using PinPoint.

 

If you do use Astropy then there are good tools in it to easily convert between image xy coords and RA/Dec in various epochs and coordinate frames.

[GeorgeCarey]

Thanks Nigel. I will have to bite the bullet and learn some Python. I have ordered a teach yourself book. Thanks for the tips.

[nakbrooks]

Happy to exchange thoughts if useful. I may be a bit ahead of you on Python in photometry but you’re streets ahead of me on most other aspects of astronomy as I’ve come to it late in life - and it appears we live only 30 mins or so away from each other. Feel free to private message me.