Jump to content


Photo

FITS image data format


  • Please log in to reply
22 replies to this topic

#1 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 13 August 2019 - 10:19 AM

Are there any experts on FITS images out there?

 

I have a friend who is a whizz at programming and have coaxed him into developing software that can take a few FITS images and search for new objects.

In other words, automated nova/supernova detection.

 

He has had some success in reading a FITS image, despite the incomprehensible information about the data structure.

 

For some strange reason, we are getting the right hand 280 columns appearing at the left - despite reading the data sequentially from the start.

 

The image is 3326 x 2504 pixels.

 

Very stretched image below:

 

Any help would be much appreciated!

Attached Files


ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#2 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 14 August 2019 - 09:08 AM

Great progress. We are now able to extract adu values from the fits file that agree with Maxim.

The hardest part is done!

 

bingo!.jpg


ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#3 LaCebra

LaCebra

    Member

  • Members
  • 43 posts
  • LocationAndalucía - España

Posted 22 August 2019 - 03:51 AM

Hi George,

 

Several years ago, I processed "fits" images in an elementary way (autostrech).
For that I used a library of functions in C#, NASA origin (?)
There was indeed a "bug" in their source code. After correction everything worked well.

If necessary, I could look for these elements.

Good luck with that.
La Cebra


Native tongue : French

Beginner : Spanish

Machine translation system : English :rolleyes: 

 


#4 Waldemar

Waldemar

    Member

  • Beta Tester
  • 289 posts
  • LocationNetherlands

Posted 22 August 2019 - 08:03 AM

Hi George,

 

Several years ago, I processed "fits" images in an elementary way (autostrech).
For that I used a library of functions in C#, NASA origin (?)
There was indeed a "bug" in their source code. After correction everything worked well.

If necessary, I could look for these elements.

Good luck with that.
La Cebra

 

Hi La Cebra,

 

You mean the 'Fits Liberator' programm from the NASA?
Can I ask what you corrected in their source code?

 

Best regards,

Waldemar


Waldemar
--------------
Mount: DDM85 Standard 12V ; Telescopes: TEC140 + 140FF + Quad TCC ; Celestron RASA; TMB 92SS + 27TVPH Cameras: ZWO ASI 174MM cool; SBIG STF8300M; ATIK4120EX; TRIUS SX-36
Filters: Astronomik and Baader NB + RGB; Lunt double stack 75 mm unobstructed signature series + BF 3400; Dutch weather and light polluted skies.  

#5 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 22 August 2019 - 08:24 AM

Thanks  La Cebra,

My friend programs in Delphi and it looks like there are no FITS libraries available.

He is having fun working it all out from scratch!


ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#6 LaCebra

LaCebra

    Member

  • Members
  • 43 posts
  • LocationAndalucía - España

Posted 22 August 2019 - 03:14 PM

@George 

 

You can see that...

 

https://fits.gsfc.na...s.html#delafits
 

 

DeLaFits

DeLaFits (free software, MIT License) is published on GitHub. DeLaFits is native for Delphi and Lazarus and is not a wrapper around another library.
Features: building, reading, editing, and rendering of a FITS primary array image.
DeLaFits supports only Single Image (one header and one data unit).

Native tongue : French

Beginner : Spanish

Machine translation system : English :rolleyes: 

 


#7 LaCebra

LaCebra

    Member

  • Members
  • 43 posts
  • LocationAndalucía - España

Posted 22 August 2019 - 03:17 PM

@Waldemar

 

No, it's CSharpFITS - a complete C# library to FITS (but with a little bug :wacko: )

 

 


Native tongue : French

Beginner : Spanish

Machine translation system : English :rolleyes: 

 


#8 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 23 August 2019 - 09:36 AM

LaCebra - that looks very interesting! I will pass that along to Lutgen. (the programmer).

 

We have a small problem reading FITS from another friend of mine.

In his header there is a keyword 'BIAS' which does not seem normal. It is not to do with bias calibration frames.

It does not seem to be a standard keyword?

In his reference image he has BZERO=33753.0 and in other images it is the usual 32768.0  (2^15)

Also his reference image has BSCALE=0.9699392681661427    and in others it is the usual 1.0000

Very odd.


ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#9 LaCebra

LaCebra

    Member

  • Members
  • 43 posts
  • LocationAndalucía - España

Posted 23 August 2019 - 10:52 AM

Hello George,
 
1 - I have a definite advantage with C# - CSharpFITS
I was able to extract the functions I needed, then decode line by line what needed to be done.
And then I copied/reprogrammed as needed (store my Bias, Darks, Flats, Lights images in SQL-Server).
I mean, I forgot everything  :D 
 
It is frankly more difficult in Delphi where sources are scarcer.
 
 
2 - Can you publish one of these images in the cloud, describe the shooting (software, camera, etc.), and finally make a synthesis of your current analyses.
I'm not sure I have an immediate answer, however....
 
 
3 - Astronomical photography is of the same nature as Lucky Luke : A long and lonely journey :)
 
 
La Cebra
 
(Translated with www.DeepL.com/Translator)

Native tongue : French

Beginner : Spanish

Machine translation system : English :rolleyes: 

 


#10 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 24 August 2019 - 08:27 AM

Good progress made. I decided to hack into the FITS files myself - the hard way!

I have no way of reading Hex values from a file so found a hex reader on the web, and a hex to decimal converter.

I can now read the 16 bit FITS files and get exactly the same adu values out as Maxim does.

I then wrote a spreadsheet that would do the task - success.

Lutgen found a bug in his code late last night - an overflow was occurring, so I think we are back on track.

 

!Bingo.jpg


Edited by GeorgeCarey, 24 August 2019 - 08:30 AM.

ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#11 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 24 August 2019 - 08:29 AM

And the spreadsheet - dealing with the peculiar values of BZERO and BSCALE:

 

RonsReferenceImage.jpg


ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#12 nakbrooks

nakbrooks

    Member

  • Beta Tester
  • 315 posts
  • LocationWiltshire (England) & Hautes-Pyrenees (France)

Posted 08 September 2019 - 10:27 AM

Catching up on this thread very late but if your friend is familiar with Python there are very extensive tools in the Astropy package and affiliated packages that would do pretty much everything you’d want. I’m sliwly building a web application, mostly using Astropy packages. This is primarily to support Photometry work.
Nigel Brooks
Stratis Observatory, Hautes Pyrénées, France.
http://www.facebook.com/stratisobservatory

#13 nakbrooks

nakbrooks

    Member

  • Beta Tester
  • 315 posts
  • LocationWiltshire (England) & Hautes-Pyrenees (France)

Posted 08 September 2019 - 01:01 PM

Astropy does require a good working knowledge of Python programming (under either Windows, IOS or Linux). It isn’t a shrink-wrapped package - it is a library of astronomy-related Python modules.

Core Astropy is documented here: http://docs.astropy.org/en/stable/

Astropy-affiliated packages are numerous. The most relevant include:

ccdproc (FITS image processing) https://ccdproc.read...s.io/en/latest/

photutils (source detection and photometry) https://photutils.re...s.io/en/stable/

Astroquery (automated querying of Vizier, Simbad and dozens of other data sources): https://astroquery.r...s.io/en/latest/

astroplan (scheduling observations based on observability and other constraints) https://astroplan.re...s.io/en/latest/

Python isn’t a difficult language to learn (easier than C# in my view). I’m 67 years old and my previous programming experience has been Excel macros and VBA but I’ve picked it up well enough in just 9 months to write a full data reduction pipeline.

Best of all it’s totally free.

I’m keen to work with other amateur Python programmers as I’m looking to open source what I’m doing.

Nigel
Nigel Brooks
Stratis Observatory, Hautes Pyrénées, France.
http://www.facebook.com/stratisobservatory

#14 nakbrooks

nakbrooks

    Member

  • Beta Tester
  • 315 posts
  • LocationWiltshire (England) & Hautes-Pyrenees (France)

Posted 08 September 2019 - 03:46 PM

Also, if you’ve not already located them:

The formal FITS file standard:
https://fits.gsfc.na...dard40aa-le.pdf

The SBIG FITS extensions (an industry standard followed by Maxim and most camera drivers):
http://www.company7....FITSEXT_1r0.pdf
Nigel Brooks
Stratis Observatory, Hautes Pyrénées, France.
http://www.facebook.com/stratisobservatory

#15 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 08 September 2019 - 04:10 PM

Thanks, I have seen most of those links. We can read the files fine now and convert to bmp.

I have been looking at astrometry. From the FITS header we have the image centre coordinates and plate scale plus rotation.

I have got a spreadsheet working to find the coordinates of an object so that the relevant area can be downloaded from PanSTARRS or Simbad.

Far from the centre of the image errors creep in. The process of converting pixel coordinates to real world is horrendously complicated.

My system uses TAN - gnomonic projection (there are about 20 different projections!)

I am not sure I am up to sorting that out.


ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#16 nakbrooks

nakbrooks

    Member

  • Beta Tester
  • 315 posts
  • LocationWiltshire (England) & Hautes-Pyrenees (France)

Posted 08 September 2019 - 04:25 PM

Yes that’s why I use Astropy. It handles all that stuff.
Nigel Brooks
Stratis Observatory, Hautes Pyrénées, France.
http://www.facebook.com/stratisobservatory

#17 ChristerS

ChristerS

    Member

  • Beta Tester
  • 110 posts
  • LocationFROSON, SWEDEN

Posted 08 September 2019 - 07:51 PM

Hi George,

 

I have a complete C# project reading the Nasa CFITSIO lib https://heasarc.gsfc.nasa.gov/fitsio/and convert it to different other formats, I can share some code. Pixinsight also uses this lib. Im developing a database driven astro solution to handle a lot of images searcheble and hadle large stacks ready Q1 2020.

 

Cheers,

//Christer

http://astropix.se/
 


Cheers,
//Christer

www.astropix.se


#18 GeorgeCarey

GeorgeCarey

    Member

  • Beta Tester
  • 376 posts
  • LocationMidlands, UK

Posted 09 September 2019 - 10:08 AM

Thanks. I posed a question on Astronomy StackExchange and everyone says 'Python'! It looks like I will have to learn some Python programming.

There is a Python interface to the Nasa CFITSIO lib so I will have a rummage through that.


ASA DDM60Pro, homemade 8" and 10" reflectors, QSI 683 camera, Astrodon 5nm filters, Pulsar dome observatory. Website: http://www.geoastro.co.uk


#19 nakbrooks

nakbrooks

    Member

  • Beta Tester
  • 315 posts
  • LocationWiltshire (England) & Hautes-Pyrenees (France)

Posted 09 September 2019 - 11:34 AM

Yes it is far and away the most popular language for professional astronomers and is well supported by the Astropy and wider Python community.
Nigel Brooks
Stratis Observatory, Hautes Pyrénées, France.
http://www.facebook.com/stratisobservatory

#20 nakbrooks

nakbrooks

    Member

  • Beta Tester
  • 315 posts
  • LocationWiltshire (England) & Hautes-Pyrenees (France)

Posted 09 September 2019 - 11:39 AM

If you do get into using Python then make sure you use the Anaconda Python framework - it is free and includes all the scientific/astronomy Python packages you’re likely to need so makes things easier.

For program writing I strongly recommend Microsoft Visual Studio Code (NOT Visual Studio itself which is much more complex). VS Code is also free and has a Python extension which does syntax checking etc.
Nigel Brooks
Stratis Observatory, Hautes Pyrénées, France.
http://www.facebook.com/stratisobservatory




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users