Smartphones can now take incredible photos that rival DSLR cameras; Thanks to the capabilities of artificial intelligence. Because mobile devices are compact, smartphone cameras cannot carry large sensors or lenses, so this challenge forces smartphone manufacturers to use computational imaging techniques to improve image quality.
The Samsung (Galaxy S23) series of mobile phones, which will be launched this year, is equipped with Qualcomm processors (Snapdragon 8th Gen 2) developed specifically for these phones.
According to Joshua Sungdae Cho, Vice President and Head of Vision R&D Programs in the Mobile Division, Samsung has deployed nearly 60 new AI models powered by the Neural Processing Unit (NPU) to enhance the capabilities of the Galaxy S22 Ultra's camera. Interview at the company in 2022.
Samsung started implementing AI models in the NPU 4 years ago, when the NPU was only running about 10 models of AI, but by 2022, Samsung phones will run 60 models of intelligence to get Best for providing camera functionality, NPU is involved in almost all camera functions.
AI models are decision-making algorithms that are trained from data sets for specific tasks, while NPUs are chips specifically designed to handle such algorithms.
What are the artificial intelligence models that Samsung uses in its phone cameras, and how has it developed artificial intelligence to improve the capabilities of its phone cameras in recent years?
1- Improve the vertical position and create a strong depth effect:
One of the artificial intelligence modes that Samsung used in the Galaxy S22 Ultra last year is the (Portrait) mode, in which the camera creates a depth of field effect, allowing you to take sharp photos with a blurred background. This mode has now become an important feature for smartphones.
Samsung developed this mode in its phone (Galaxy S22 Ultra) earlier this year by improving the front camera with artificial intelligence to ensure the finest details and purest colors in low-light conditions.
It also introduces depth mapping, which allows you to focus on portraits faster, and portraits of people in low light (portrait) mode become more accurate with AI-powered in-depth analysis.
Some cameras use depth sensors, but it is difficult to control the depth effect by the camera alone, so Samsung uses an artificial intelligence model that is trained on a large number of images, backgrounds, and various depth-related objects.
2- 100x digital zoom:
(Galaxy S23 Ultra) The phone camera supports 3X and 10X optical zoom and 100X digital zoom.
Oh yes; This is not the first time that Samsung has introduced 100x digital zoom, it was first introduced in the (Galaxy S20 Ultra) phone 3 years ago, but with the growth of the new d model, Samsung continues to impress users with the processing power provided by the 4nm processor in the latest Her phones (Galaxy S23 Ultra).
Processing time is critical for features like 100x zoom in and out, and Samsung has the computing resources to manage that, with some operations done by the NPU and others by the GPU, all in a manageable time frame.
For images taken at digital zoom greater than 30x, the camera takes 10-20 background images in a single shot, then stitches multiple images together before performing additional post-processing to determine which details to enhance.
3- Improve night photography:
The Galaxy S23 Ultra's 200MP camera sensor can combine up to 16 pixels into a single larger pixel to increase the amount of light the camera captures, resulting in brighter, more detailed photos in low light. Whereas the 108MP camera sensors in last year's phones can only pack nine pixels into one to better absorb light in low-light conditions.
This technology, combined with a trained AI model, can reduce noise and provide a reasonable increase in brightness in night lighting.
Samsung trained this AI model using various photos taken in night light, ranging from photos taken with DSLR cameras to photomontages Samsung created to professionally increase brightness and reduce noise.
And on the new Galaxy S23 phones, Samsung has fixed visual issues that negatively affect low-light photos with a new AI-powered image signal processing algorithm to improve detail and sound.
4- Take the same photos with a DSLR camera and edit them in RAW format:
Compressed image formats such as JPG and HEIF limit image quality and users' ability to adjust fine details, but the new cameras in the Samsung Galaxy S23 series provide resolution and quality at the highest frame rates in the series' history, including the following features: support for higher resolutions and multiple RAW image capture Uncompressed 16-bit frames plus advanced astrophotography and multiple exposures.
Shooting in RAW format is a great choice for mobile photographers because it provides realistic colors and details that can't be captured with a smartphone.
In early 2022, Samsung launched Expert RAW, an advanced camera application that brings a host of new photo features and controls designed for the flagship Samsung Galaxy phones.
Aimed at experienced photographers, Expert RAW builds on RAW format recordings, complemented by greater dynamic range that provides more detail in the light and dark areas of an image and offers advanced editing capabilities.
But this year, you'll find an (Expert RAW) mode built into the phone's native camera app, which should make (S23 Ultra) a suitable option for those who enjoy creatively editing their phone's photos. Samsung is also making Adobe Lightroom the default app for editing Expert RAW files on its phones.
Samsung uses NPU and AI to render 16-bit RAW images in its phones, and since the camera collects 20 shots in the background, users on the Ultra model can use up to four lenses at once.
DSLRs have larger sensors and can take RAW images without noise, but until now, smartphone cameras have struggled to imitate low-light situations due to the smaller sensors. But according to Joshua, with the processing power of the Galaxy S23's Neural Processing Unit, the 16-bit RAW images match those captured by professional DSLR cameras.
finally:
Computer Imaging, i. h. The technology used to capture and manipulate digital images is growing rapidly as the power of neural processing units in smartphones doubles every year due to advances in processors, a trend that is expected to continue for at least the next five years. We hope that the cameras in smartphones will continue to improve over the next few years.
The hottest topic in computer imaging is the multi-camera technology, so researchers and companies are always looking for the best multi-camera setup because cameras are the main selling point of smartphones today.
So we expect that with the development of artificial intelligence that we are witnessing today, in the next few years, we will see all smartphone camera functions managed by hundreds of algorithms provided by the NPU.