How We Built the Merge Records Website to Handle All Those Digital Music Files

Merge Records, an independent record label in Durham, NC, is the home of critically acclaimed and popular artists like Arcade Fire, Spoon, the Magnetic Fields, Neutral Milk Hotel, and many more. Theirs is also one of our latest website builds — one that presented a few interesting development challenges.

While we’ve written at length about the overall Merge site build here, we thought it would be helpful to look specifically and in greater detail at the development side of what was a particularly complex project.

A Few of the Major Requirements

Some of the major requirements included:

• An online management system for media assets, including over 200 artists, 500 albums, and 5000 individual songs.
• A digital store offering the sale and download of both lossless FLAC and high quality 320kps MP3 files.
• Streaming from the site’s “Merge Radio” feature.
• The ability for shoppers to “preview” a 20-second clip of each track/album available for purchase.
• Integration with Merge’s existing order-fulfillment system to automate both order processing and the syncing of tracking numbers and shipping statuses.

All of this meant that the site, when completed, would have to be able to handle a huge amount of data every day. The solution needed to be not only manageable from a server-resources standpoint, but also offer a streamlined workflow for the administrators at Merge.

The end solution is a multi-tiered system that incorporates Merge’s digital asset partner (CI), as well as Rackspace’s cloud file storage solution. Tying it all together is the Merge website itself.

How it All Comes Together

As part of the NewfangledCMS, Merge manages their artist, album, and song information the same way they would any other website content. When setting up a new album entry, they can check a box to schedule a file transfer from CI, and all the files for previews, streaming, and download will be transferred automatically, saving Merge the trouble of manually uploading the dozens of separate song and album files used by the site. Each night, the site downloads song files for new albums entered in the CMS using the application programming interface (API) web service provided by CI to a temporary location on the web server. The site then creates a zip of the FLAC and MP3 files for sale as a full album.

Next, the site uses the API provided by the cloud hosting service to transfer those zipped albums and individual song files for sale to a secure location at the cloud hosting service, and to a content delivery network (CDN) for the preview and streaming files. Finally, the site updates its own records with those new locations and deletes the temporary files. If Merge wants to use a file that can’t be handled by the automatic transfer — for instance, if they’re selling an album that includes bonus materials not provided by their encoding service — they can upload their own file to the cloud hosting directly and enter that new file location in the CMS.

When the customer buys a digital copy of a song or album, the site creates a download code for them, which they can use a limited number of times. When the customer uses the code to download a song or album, the site generates a temporary URL at the cloud service from which the user’s browser downloads the album or song.

Merge can track those downloads to see which ones were used and when. They can also reset the existing codes or generate new ones as needed to handle customer support. In addition, they can use the site to generate lists of download codes to be included with LP sales, so that customers who buy vinyl can also download the digital media that is often offered as part of the vinyl package.

To handle the fulfillment of online orders, the site also provides it’s own XML-based web-service API, which is used by Merge’s order-fulfillment software to download orders submitted by customers through the site, as well as updating the site’s record with shipping dates and tracking numbers. In architecting this, we worked closely with the team at Merge to create a system that emphasized the parts of their existing software that were working well, while leveraging new capabilities made possible by the new website.

The End Result

Utilizing web services is often part of any website project — however, in the case of Merge, the end result is a complex, unified system, tailor-built to provide the most automation and flexibility for Merge’s business. In addition to the custom integrations described above, Merge enjoys the CRM and marketing automation integration, which together form the lead development ecosystem that Mark and Chris wrote about in The Future of Web Development. The website, therefore, sits at the center of a larger ecosystem of web services managing the flow of data and digital assets between them and the site’s users and customers. The end result is, in our opinion, a very powerful system, indeed.