Samsung Researchers Explain How Nightography Delivers ‘Epic Night Shots’

Samsung’s Nightography feature lets you take stunning photos in the dark. If you love taking photos at night, but hate how they turn out blurry and dull, Samsung has a solution for you. The company’s Nightography feature was first introduced in the Galaxy S22 series. And enhanced for the Galaxy S23 series. It uses advanced AI and computational imaging to deliver vivid, highly optimized photos even in low-light conditions.

According to Samsung’s researchers, Nightography combines multiple frames of different exposures. And applies noise reduction and tone mapping algorithms to make a single image that maintains details and colors. The feature furthermore adapts to different scenarios, such as cityscapes, landscapes, or portraits, and adjusts the brightness and contrast accordingly.

Nightography is the art of capturing the hidden beauty of the dark, not just making it look a little less dim. It’s about capturing the mood and atmosphere of the night with realistic colors and natural skin tones, said Dr. Lee Seung-hwan, head of Samsung Research’s Computational Photography Lab. He further added that their aim was to provide users with the capability to capture breathtaking night photos that they can confidently share with their loved ones.”

John Seok-Jun Lee and Narasimha Gopalakrishna Pai, lead the camera and AI teams at Samsung in America and India. They talked about how they worked together to make Nightography transpire.

Researchers take on the revolutionary S23 Feature

John Seok-Jun Lee said Nightography lets users obtain definitive photos in low-light environments. The S23 camera collects a series of images with different brightness levels, then applies multi-frame processing (MFP) technology. This technology examines the images and eliminates the frames with noise or blur. And then merges the remaining frames into a single clear shot with HDR. Then, an AI-powered ISP algorithm on NPU improves the photo’s color, tone, and details.

In addition to that, Narasimha Gopalakrishna Pai said that the image quality of Nightography on the Galaxy S23 has seen a remarkable improvement in brightness. Also without compromising on controlling noise, clarity, and saturation around light sources such as neon signboards. The SRI-B team utilized the Snapdragon 8 Gen 2, which is exclusive to the Galaxy S23 series. Thus, to optimize the multi-frame pipeline and accelerate Al models. This resulted in faster processing and lower memory usage than the Galaxy S22.

Also Read- Samsung One UI 5.1 Comes to Current and New Galaxy S23 Series

Successful R&D Collaboration For Nightography Between MPI and SRI-B

Through the analysis and merging of multiple captured inputs, improvements are made to the Signal Noise Ratio (SNR) and dynamic range. While optimizing blur caused by camera or object motions. To further enhance photo details and remove high noise levels, SRA MPI collaborated with MX Visual Solution Team. Therefore, to develop and optimize the Al image signal processing (ISP).

SRI-B was tasked with the development and optimization of the algorithm pipeline, as well as accelerating Al models using all available compute units on the platform, including multicore CPU, NPU, and GPU. Furthermore, SRI-B designed an Al denoising module that effectively reduces motion blur in the final output. Additionally, SRI-B created Night Portrait, which allows the Galaxy S23 selfie camera to capture stunning night photos.

Also Read- With Galaxy Experience Space, Samsung Brings Epic, In-person Experiences

Overcoming the Rising Challenges

John Seok-Jun Lee said when asked how did they overcome challenges they faced while developing the amazing feature. He said that Smartphone cameras capture significantly less light at night than during the day or in indoor scenes, resulting in darker input frames with higher levels of noise, blur, and a narrower dynamic range. Simply increasing the brightness of these frames can lead to undesirable effects such as noise, blur, and color distortion. This is where Nightography’s AI MFP technology comes into play, executing complex analysis tasks to determine the optimal tuning points and leveraging an Al-powered ISP to analyze each pixel and produce spectacular nighttime pictures.

Moreover, Narasimha Gopalakrishna Pai added that Nightography uses a multi-frame method to capture and process numerous frames. Moreover, discarding those that possess blur and artifacts. To guarantee that the user has a seamless experience when capturing shots, the SRI-B team produced a heterogeneous computing pipeline that leverages all available calculate units on the CPU, NPU, and GPU for fast MFP processing. Additionally, SRI-B overcame the challenge of developing and optimizing high-resolution Al models for smartphones. This is accomplished by working closely with SRA MPI and MX Visual Solutions teams.

Also Read- The Galaxy Book3 Series, Samsung’s Latest Flagship Windows 11 PC Lineup

Future Goals For the S23 Series and Upcoming Series

SRA MPI plans to address the challenge of low-light photography in high mega-pixel cameras by leveraging next-generation Al MFP technology. Additionally, the team aims to improve the quality of video Nightography to match that of photo Nightography. The ultimate goal is to enhance image quality and user experience by continuously improving Nightography and Expert RAW for all image resolutions. Furthermore, the team plans to explore the potential of applying Nightography to other premium devices beyond the Galaxy S23 series.

Also Read- Samsung Electronics Exhibits Award-Winning Machine Translation At WMT