The Sony "Star Eater"

What is the Sony "Star Eater"?

The Sony "Star Eater" is a spatial filtering algorithm that is applied to the data in order to remove hot pixels in longer exposures. However it can also remove complete stars or the centres of stars from astro-images. Early Nikon digital cameras applied an identical spatial filtering algorithm which became dubbed "Star Eater" by the asto-imaging community. The star eater became infamous amongst astrophotographers and they avoided Nikon cameras. When it was discovered that Sony cameras were doing the same thing, it was natural to call it "Star Eater" again.

Which Sony cameras are affected?

All Sony full frame cameras (A7, A7R, A7S, A7II, A7SII, A7RII, A9) and A6000 series cameras (e.g. A6000, A6300, A6500) are affected by the "Star Eater" spatial filtering in Bulb mode. The duration of the Bulb exposure is unimportant - a short or long Bulb mode exposure will be affected by the "Star Eater".

In addition, a notorious firmware update to the Mark 2 versions (A7RII, A7SII but not the A7II) introduced the star eater to all exposures 4 seconds and longer. The Mark 2 models have IBIS image stabilization and the purpose of the update was probably to address the issue of increased noise in longer exposures. Because the sensor has to be free to be move (which is how stabilization works) it is much more difficult to conduct away heat from the sensor. Hence the temperature rises and so does the thermally induced noise which particularly affects longer exposures.

The star eater spatial filtering was extended from Bulb mode to all exposures 4 seconds and longer in the v1.10 firmware update to the A7RII.
The accompanying description was as follows:
v1.10 : "Decreases chroma noise when Long Exposure NR (Noise Reduction) is set to Off. (Note: The noise does not occur when Long Exposure NR is set to On."

However, this was not spotted until update v3.30 was released: version 3.30 on the A7RII is affected.
v3.30 was widely blamed for causing the issue. It is not known which firmware update caused the A7SII to be similarly affected.

After the complaints, Sony released a new firmware version (version 4.0) for the A7RII which reduced the effects of star eater filtering in the green channel but the red and blue channels were affected just as badly as previously.
At the same time, Sony released a new firmware version (version 3.0) for the A7SII which is thought to show a similar improvement in the green channel but to the best of my knowledge it has not yet been conclusively demonstrated.

Camera by camera summary

   Sony A7, A7R, A7S, A7II:    Star eater affects Bulb mode only.

   Sony A7RII:
      Firmware v1.0    Star eater affects Bulb mode only.
      Firmware v1.10 - v3.30    Star eater affects Bulb mode and all exposures 4sec or longer
      Firmware v4.00    Updated star eater algorithm affects Bulb mode and all exposures 4sec or longer

   Sony A7SII:
      Firmware v1.0    Star eater affects Bulb mode only.
      Firmware v2.00    Unknown
      Firmware v2.10    Star eater affects Bulb mode and all exposures 4sec or longer
      Firmware v3.00    Released at the same time as A7RII v4.0 and is probably the updated algorithm

   Sony A9:
      Star eater affects Bulb mode and all exposures 4sec or longer. It probably uses the updated star eater algorithm found in A7RII v4.0

   Sony A7RIII:
      Uses the updated star eater algorithm found in A7RII v4.0

References:
It was A7RII v1.10 that caused it!!
A7II is unaffected at 4sec for all firmware versions
A7RII version 3.30 is affected at 4sec
A9 is affected at 4sec

What are the effects of the Sony "Star Eater"?

The purpose of the "Star Eater" spatial filtering is to reduce the image noise caused by hot pixels - these are image pixels whose value is larger than surrounding pixels.
Consider the diagram below of the colour filter array that overlays the sensor:

Taking a red pixel, for example, it has 8 other neighbouring red pixels - not immediately adjacent but nearby: left, right, up, down and 4 diagonally. If a red pixel is brighter than its 8 neighbouring red pixels then the spatial filtering algorithm replaces its value by the largest of the values in its 8 neighbours. This reduces its brightness and makes it less obvious in the image. The same argument applies to the blue pixels. The green pixels work similarly but the 8 neighbouring pixels are left, right, up, down and the diagonal pixels that are not immediately adjacent.

Now suppose the camera has an excellent lens that can focus the light of a star onto a single pixel. It will make this pixel brighter than its neighbours. It looks like a hot pixel and the spatial filtering algorithm will then truncate it down to the level of its less bright neighbours. So the star disappears, it is eaten.

A single pixel star is probably unrealistic so let's consider a star whose light is spread over a group of 4 pixels, a 2x2 block:

The effect of the spatial filtering is that the bright red pixel will have its brightness reduced to the brightness of neighbouring red pixels. The bright blue pixel will have its brightness reduced to the brightness of neighbouring blue pixels. Both bright green pixels will also have their brightness reduced to the brightness of green pixels outside the star's area. The result is that the whole star disappears.

Now consider a star whose light is spread over a 3x3 block of pixels. Here are the before and after diagrams:

The central pixel of this particular star is red. As before, the spatial filtering will reduce its brightness. However in this example the bright green and blue pixels are "protected" by other bright green and blue pixels within the star's area. The result is a star with its bright centre punched out:

Here are the before and after diagrams for a star whose light has spread over a 4x4 block of pixels:

The central 2x2 bright core of the star has its brightness reduced to that of the dimmer outer regions.

In summary, single pixel stars and 2x2 pixel stars are completely erased but stars that spread beyond 2x2 pixels will survive, possibly with their centres removed. Stars that spread further will have their central peak dimmed.

The following example is the crop of a Bulb mode 5 minute exposure taken with a Sony A7S camera on a Takahashi Epsilon telescope, shown at a 200% scale so the individual pixels can be clearly seen:

The stars with centres punched out are quite obvious. Of course it's not possible to tell how many smaller stars have been completely removed. The other effect is a change in the star colour - with the brighter pixels "knocked out" the overall colour balance of the star changes leaving impossibly weird colours for stars.
Note that this example shows the effects for a fairly long focal length, so all the stars have a certain size. With a good quality wide angle lens the stars will be smaller and much more at risk of being totally erased.

Here are another couple of examples of a single star taken with the same Sony A7S camera on a Takahashi Epsilon. A normal exposure (on the left) was immediately followed by a Bulb mode exposure of identical duration (on the right). The images are views of the raw data before being converted into a colour image:

Example 1:

Example 2:

The next example is a long exposure dark frame i.e. an exposure taken with the lens cap on. Again this shows the raw data before being converted into a colour image. The data should be completely random but the spatial filtering has rendered it far less random:

Notice the pairs of pixels with identical brightness. This is where the spatial filtering has reduced a bright pixel down to match the brightness of one of its 8 neighbours. Close examination of raw data makes it possible to determine that spatial filtering is taking place even without shooting an image of stars.

The next example is a simulates the effect of the "Star Eater". The original image is a non-Bulb exposure taken with the Sony A7S camera on a Takahashi Epsilon telescope. The spatial filtering algorithm was run on the raw data so an exact before and after comparison can be made. Again the image is shown shown at a 200% scale so the individual pixels can be clearly seen:

Notice the attenuation in brightness of many stars and how many stars are turned into rings with weird colours. Few stars have actually disappeared because a combination of the fairly long focal length (500mm) combined with atmospheric turbulence means that the starlight is spread over an area larger than a 2x2 pixel block. One other point of interest is the bright stars that have saturated the sensor. Where a star has saturated, each pixel has a neighbour that has also saturated and so the spatial filtering has no effect. With a good quality wide angle lens, a large number of stars will be no larger than a 2x2 pixel block and will therefore be totally erased.

Diagnosing and Reverse Engineering the spatial filter

More information on how the algorithm is diagnosed and reverse engineered is here: Diagnosing The Sony "Star Eater"

Is the "Star Eater" a median filter?

The "Star Eater" spatial filter is sometimes referred to as a median filter. This is not technically correct. A median filter is certainly a type of spatial filter but its effects would be far more destructive.

For the technically minded, the formula describing "Star Eater" is quite straightforward to understand. Consider the following diagram:

For any given pixel V0 (which will be red, green or blue), the values of the 8 neighbours V1-V8 are examined and the value V0 is replaced if it is higher or lower than the highest or lowest of those 8 values, as follows:

If V0 > Max(V1,V2,V3,V4,V5,V6,V7,V8) Then V0 = Max(V1,V2,V3,V4,V5,V6,V7,V8)
Else
If V0 < Min(V1,V2,V3,V4,V5,V6,V7,V8) Then V0 = Min(V1,V2,V3,V4,V5,V6,V7,V8)

Note that pixels V1-V8 will always be the same colour as pixel V0 i.e. all red, all blue or all green.

Why are many night sky photographers unaffected?

Many examples can be found of "before and after" images where apparently no stars are eaten.
On closer examination it is usually discovered:
1) The lens optics or bad focusing have resulted in stars much larger than a 2x2 pixel block.
2) The image has been taken with a fixed tripod (i.e. not a tracking mount) so the star trailing has created elongated stars longer than a 2x2 pixel block.

Regrettably this has resulted in a certain amount of misinformation and confusion about the "Star Eater" issue.

How does the "new" Star Eater algorithm differ?

The improvement to the "Star Eater" spatial filtering algorithm was made in firmware v4.0 for the Sony A7RII. It is believed it was also implemented in firmware v3.0 for the Sony A7SII but I haven't yet seen any proof. The improvement makes a difference only for the green pixels. The blue and red pixels are affected just as badly as previously. The effect is that more stars survive than previously but these "new" survivors are predominately green in colour. A detailed description can be found Sony Star Eater v2.

Which kinds of astrophotography are affected?

Although it is possible for everyone to see the evidence that a spatial filter is at work, the effect on real astro-images remains an area of disagreement.

Some people claim to have never experienced the effects while many others see the "Star Eater" as a show-stopper. It all depends on the expectations of the photographer, the equipment used, shooting techniques, processing techniques and finally how the resulting image is displayed.

Some factors to be taken into account are the following:

Star Image Size If the lens/telescope optics or slight star trailing result in star sizes 3x3 pixels or larger then stars will not be completely erased, though they are likely to be attenuated in brightness. Stars are most likely to be affected when using very sharp optics together with a tracking mount because this keeps the star images tight.

Star Brightness Brighter stars tend to occupy more image pixels and thus escape problems. It is the dimmer background stars that are most likely to be affected.

Pixel Size For a given lens/telescope, a camera with larger pixels (e.g. A7S/A7SII) is going to be more affected than a camera with smaller pixels (e.g A7R/A7RII/A7RIII) because the stars will occupy fewer pixels.

Deletion vs Damage By definition, deleted stars will not be visible and their absence may not be noticed. Damaged stars with their centres removed or with very strange colours will be more noticeable. Attenuation of star brightness may not be noticed.

Display A 36Mpixel image scaled down for display on a single screen is unlikely to show any obvious problems. On the other hand, a cropped image or 12Mpixel image displayed on a high resolution screen will clearly show any damaged stars

Processing techniques Some processing techniques used by serious astrophotographers such as image stacking and Bayer Drizzle are likely to preserve and emphasize the damage caused to stars. Stacking a large number of dithered exposures (i.e. exposures with random offsets) is likely to average out any strange star colours resulting from the original star eater algorithm. On the other hand, stacking is likely to emphasize the "green star" effect of the A7RII v4.0 algorithm. Bayer Drizzle, which is very good at improving image resolution during stacking of multiple dither exposures, will unfortunately preserve the holes in stars.

From a personal viewpoint my goals for astrophotograpy are to produce high resolution images with natural star colours. Any kind of star damage is completely unacceptable to me, especially since I often use Bayer Drizzle for stacking exposures. Currently I use a Sony A7S which allows me shoot exposures up to 30sec without being affected by the "Star Eater". 30 sec is the upper limit without using Bulb mode. However, this requires me to take hundreds of exposures to be stacked into the final image.

Conclusion

We have seen how the "Star Eater" spatial filtering affects images of stars. Single pixel stars and stars contained within a 2x2 block are completely erased while stars that are slightly larger typically have their centres punched out or dimmed. It's very much an individual's decision whether or not this is likely to affect to affect their own photographic goals.