In 2017, three orphans from Sinaewon Orphanage in Goyang, South Korea came to Los Angeles on a sponsored trip to experience a new life.  During this trip, Sarah Chung saw the orphans give an impromptu performance on violin, flute and cello. After the performance, she met with a representative of the orphanage and learned that the orphanage had an established music program but lacked instruments. Through this encounter, the idea for With Music was born. She invited the orphans to perform at a recital at Los Angeles Conservatory of Music along with some of its students. This first charity concert included an auction for baseballs signed by pitcher Hyun-Jin Ryu of the Los Angeles Dodgers. The orphans were gifted with presents and money for their trip. After their return to South Korea, the work of With Music continued.

With Music seeks to improve the lives and communities of others through music. Musical instruments have been donated to Sinaewon Orphanage and other organizations are also being assisted by the fundraising efforts of the organization. With Music operations include a series of charity concerts and philanthropic activities. In addition to assisting underprivileged communities, the student volunteers of With Music have greatly benefited from taking active leadership roles within the organization. Several participating professional musicians have donated their time and performances to assist with fundraising as well. With Music formalized its operations in 2019 and incorporated as a recognized 501(c)(3) non-profit corporation.