Subscribe to the blog
Follow on Facebook
Data storage and management is no mean feat. And contrary to popular opinion, it takes up a huge part of the workflow in our studio. The work doesn’t just end when we click and deliver the photos. After that begins storing, backing-up, managing, organising and archiving all that data. And believe us, it’s a LOT of data. As you know, Indian weddings are full of rituals and go on for days and days! Which means that there’s loads of data to be handled. To give you an idea, we generate about 2-3TB of data per wedding. Let’s say we approximately shoot 30 weddings a year, that leaves us with 60-90 TB of data per year. Now with our tendency to store old data in case, a client needs it again, over the past ten years data management has been a huge pain-in-the-you-know-what.
We’ve already spoken in detail about the data storage workflow we follow in our studio. We back it up in three different mediums across two different locations. But this is something we have been doing for years and years. And of late, we have started to realise that we need a new system for data management and storage. Our old way works for sure, but it still renders some level of vulnerability to the data. Hard drives crash and fail, it’s just a question of when. It’s a hard truth we have to accept. And to be honest, it also gives way to a LOT of hard drives. We have about 70 hard drives of various capacities and lots of rotating pen drives. It’s a lot of hard drives and it’s a lot of work keeping them.
Choosing the right storage system can be a challenging process because of so many options available in the market today. We explored a lot of options while figuring out the one that would work best for us – Dropbox unlimited, private cloud, manual storage and backup (which is what we did), fire-safe vaults, NAS and more.
The case against cloud storage: The online storage seemed like a cost-effective and safe method but it didn’t work out for us in the long scheme of things. We did the math – even if we had a fast internet connection, say 100MBPS, which is as good as it goes theoretically, it would still take DAYS for data from one wedding to get backed up. And while it’s possible to do it, it’s still a painfully long and manually intensive process. One system would always need to be online, monitored by someone and if it failed or stopped in between, then we start all over again. Also, even if we were able to do it manually, most Indian internet connections have a fair usage policy, and so after a few weddings the speed would fall down and we would be back to square one. let’s say for argument’s sake that we were able to upload all the data we have ever owned onto the cloud, downloading it again if we ever need it would be a pain.
To add to the above, the time it takes to upload data doesn’t only depend on our internet speeds. A lot of cloud storage servers have painfully slow speeds and if the server’s speed is slow, then we’re stuck again with loads of data to back up.
Well, the two biggest things we wanted to rid ourselves off of is the data vulnerability in hard drives and the hoarding of those said drives. Data is our biggest capital and keeping it safe our biggest priority. But we also wanted a system where all the data was kept in one place without the fear of loss or mismanagement. And to add to it, we needed a process that was faster and less time-consuming than what cloud storage could give us.
Data sharing with the team was also a huge hassle before. Anytime we needed to share a document, video or a photo, we had to move it through hard drives. If one photographer was working on culling a wedding-set, another on editing and a third on the album, a hard drive had to be passed on every time we wanted the process to move ahead. Version control was a nightmare. Also, only one person could work on it at a time. And sure, we have Dropbox and Google drive systems set up for sharing data, but with the amount of data we have, we need to upgrade their storage capacities very often. That’s neither cost-effective nor efficient. Furthermore, the Dropbox and Google drive sharing is good for smaller files, and perhaps even a few GB’s but not for larger data. So we needed a better system of storing and sharing data.
The case for a NAS: So when Synology and Seagate came to us with a solution, we jumped at the chance of exploring something new and efficient for our workflow. We’ve always known about NAS but always hesitated at the chance of exploring its system of storage. But there was nothing to worry about really, once we understood how it worked, it made total sense! It ticked off all the boxes in our list – our data was as safe as it could get, everything was in one place and the transfer rate was much faster.
Before we dive into how NAS has made our workflow more efficient and how we have integrated it into our studio workflow, let’s talk about what it actually is.
The full form of NAS is simple – “Network Attached Storage”. And this gives you a basic idea of what it constitutes. It’s basically a pile of hard drives in one storage system connected to your computer (home or office) network. One can access files on the NAS through a computer, tablet, or smartphone over the Internet or a local network. NAS runs autonomously as it uses its own CPU and memory to manage storage. The NAS acts as a central storage hub through which multiple computers can access files.
A NAS is a part of a networking ecosystem (something we didn’t realise until more tech-savvy people turned their noses up at us!). For a NAS to work efficiently, you need a fast network infrastructure to back it up. Till now, we were working with the regular WIFI that the default internet router came with. This turned out to be painfully slow, so we started reading up – and assuming you are a team of multiple photographers and editors, each with a work computer of your own, this is what you’ll need to set up a NAS in hour home or office environment.
#1 A Gigabit router: This is the newest type of router, and provides much higher data read and write speeds than regular routers. In fact, there is even talk of a ten-gigabit router, but that’s still something we need to try.
#2 A Gigabit network switch: A switch is a network device that divides network traffic equally in all directions. You need this because your typical router comes with limited ports – and you’ll need more than the 3-4 that are included.
#3 CAT 6/7 ethernet cables for all systems accessing it: The last link in the chain, a good quality network cable will ensure high speeds throughout (and work much faster than accessing data over a WIFI network).
NAS is a one-point storage system. It eliminates the need for triple back-ups because this one system serves it all. So there’s no redundancy of data in multiple drives. We don’t need it because our data is pretty safe in there, but we still maintain another back-up in a regular hard drive at a different location because we’re paranoid about data loss. It is, after all, our most important capital.
All our working files – documents, PDF’s, jpegs, video files, project files, presentations, and any other files are all saved in the central server. This acts just like a Google drive – anything we need to share with a colleague, we transfer on to the central server. So anyone in our company can access the files through our private server and it has led to greater collaboration between our employees. We no longer need to pass around pen drives and hard drives or run the risk of losing or mismanaging data. NAS has made our inter-office data sharing very convenient. We can also assign user permissions to certain folders so that we don’t cross wires between each other, every person has access to data related to their department.
The NAS also serves as a private central server for all our photo data. Instead of the files being passed around on a drive from the person who culls images, to the photo editor and then the album designer, all the raw files sit safely at one place on the server. Each person working on those files can access them directly from the server and there’s no risk of data mismanagement. And what’s more, Lightroom can directly access the raw files from the server and we don’t need to connect hard drives or copy the files on our system. So when one person finishes their work, the next person can pick up where it was left off without any hassles.
NAS has also changed the way we deliver to clients. Delivering high-resolution photos and videos to clients has definitely been a challenge because of their large size. Drive links expire or eventually, because of storage space limitations we have to delete some data to make room for more. But with the NAS, now we can share files of any size in any part of the world sitting on our private server. The client gets a direct private link for download.
We not only get the benefit of having our work files safe on our server, but also periodic back-up of the work-in-progress projects. With NAS, there is a built-in snapshot feature that automatically takes a copy of the files and projects we’re working on. So in case your system or your software crashes or hangs up before our work is saved, which happens more often than we all would like, you have a version of the project saved up on the server and can pick up right where you left off. Currently, we’re using Apple’s brilliant Time Machine back-up system in conjunction with our NAS. The back-up system is automatic and the data sits safely on our server.
Once the files are on the NAS server, we can access them from anywhere in the world with an internet connection. And the reverse is true as well. We can even upload files on the NAS through an internet connection when on a shoot. That way we don’t have to worry about data loss or hard drive malfunction on the field and we have two back-ups – one on the field hard drive and one on the NAS back home.
Synology comes with a robust app-store of sorts called add-on packages that extend the functionality of the NAS. You can download and install the packages that you specifically want according to your needs. There are a lot of packages available of which we have the Photo Station, File Station and Synology Drive Server installed on our NAS. As we write this, we are exploring more packages that can aid us and help make our workflow easier and better. We’re also scoping out the option of installing the Surveillance Station package to allow back-up and other services for our in-office security camera system. You can check out the packages and know more about what functions each of them offers on Synology’s website.
There are a lot of options when it comes to choosing the NAS station. The good folks at Synology helped us figure out the DiskStation most suitable for our work. We have a DS1019+ DiskStation with 5 hard drives of 8TB capacity each. Choosing the right partner for the drives was a no brainer. We use the Seagate Ironwolf drives and we trust them to keep our data safe. The NAS also allows you to scale up. As the amount of data you need to warehouse grows, you can increase the storage capacity of the drives.
Now if you’ve ever looked at getting a NAS system for your office, you must have come across the term “RAID”. What is it? Well, we had to spend some time ourselves to understand what it means in general and what it means for the storage system we wanted to implement. And here’s a quick explainer.
RAID (Redundant Array of Independent Disks): The NAS is a connected system of hard drives and is susceptible to fail like any other even though the chances are minor. This is where it’s RAID system comes in. There are two kinds that you need to understand – RAID 1 and RAID 5.
What RAID 1 does is mirrors the disks. So if you have 4 hard drives in your NAS system, two would be exact copies of the other two. While this does give you back up, it renders half of your storage space unusable because the other two drives are just mirroring the data you copy on the main two drives.
RAID 5 works a little differently though – it distributes data across multiple hard drives to protect the said data in case of a hard drive failure. Let’s say, one of the four hard drives fails, the other three have the ability to rebuild and recover the files from the damaged hard drive. This gives you three drives worth of storage space.
We have the Synology Hybrid RAID with data protection for one drive fault tolerance. So if one of the five hard drives fails, the other four will work to recover and rebuild the lost data. That gives us about 29.08TB of usable storage out of 40.
Our two cents? Yes, absolutely. If you’re a company with huge amounts of data inflow and outflow and have a lot of inter-office sharing required, you should definitely set up a system. But remember that it won’t be cost-effective if you own at most, 3-4GBs of data at any point in time. The NAS is built for larger data storage. And it has changed the way we keep our data for the better. For a photography studio our size that deals with huge amounts of data every day, it’s a necessity.
Any questions regarding data storage or the NAS? We’ll be happy to help you out!
Disclaimer: This post has been written in association with Synology and Seagate but the opinions are strictly our own.