May 31, 2013—June 2013: A device is bought to make back-ups of Clinton’s private server, but a Clinton company makes clear it doesn’t want any back-up data stored remotely.

130531austinmcchorderiktraufmannhearstctmedia1

Datto Cloud engineer Charles Lundblad (left) chats with CEO and founder of Datto, Austin McChord, at the firm’s Norwalk, CT headquarters. (Credit: Erik Traufmann / Hearst Connecticut Media)

On May 31, 2013, Platte River Networks (PRN) takes over management of Clinton’s private server. On the same day, PRN buys a Datto SIRIS S2000 data storage device, which is made by Datto, Inc. Over the next month, this is attached to Clinton’s server to provide periodic back-up copies of the data on the server. PRN sends a bill for the device to Clinton Executive Service Corp. (CESC), which is a Clinton family company.

CESC employees work with PRN employees on how the Datto device is configured. Datto offers a local back-up and a remote back-up using the Internet “cloud.” CESC asks for a local back-up and specifically requests that no data be stored in the Internet cloud at any time.

However, due to an apparent misunderstanding, back-up copies of the server will be periodically made both locally and in the cloud. This will only be discovered by PRN as a whole in August 2015. (US Congress, 9/12/2016)

However, despite internal PRN emails from August 2015 indicating many PRN employees didn’t know about the Datto cloud back-up until that time, the FBI will later find evidence that an unknown PRN employee deleted data from the cloud back-up in March 2015, meaning that at least one PRN employee had to have known about the cloud back-up by that time.

Early June 2013—Early July 2013: Clinton’s server is relocated and then replaced by a new server, but the old server keeps running.

After Platte River Networks (PRN) is selected to manage Clinton’s private email server on May 31, 2013, the company decides to immediately relocate the server and then also replace it with a better one.

130601PlatteRiverFoundersPlatteRiverNetworks

The founders of Platte River Network: Brent Allshouse (left) and Treve Suavo (right). (Credit: Platte River Networks)

PRN assigns two employees to manage the new server (which will be the third server used by Clinton). The FBI will later redact the names of these two employees, but it is known that one of them works remotely from his home in some unnamed town and will handle the day-to-day administration of the server, and the other one works at PRN’s headquarters in Denver, Colorado, and handles all hardware installation and any required physical maintenance of the server. Media reports will later name the two employees as Paul Combetta, who works from Rhode Island, and Bill Thornton.

The employee at PRN’s headquarters (who logically would be Thorton) works with Clinton’s computer technician Bryan Pagliano to help with the transition. Around June 4, 2013, this person is granted administrator access to the server, as well as any accompanying services.

130601EquinixLogo

Equinix Logo (Credit: public domain)

On June 23, 2013, this person travels to Clinton’s house in Chappaqua, New York, shuts down the server, and transports it to a data center in Secaucus, New Jersey, run by Equinix, Inc. This older server will stay at the Equinix facility until it is given to the FBI on October 3, 2015.

The PRN headquarters employee (still likely to be Thornton) turns the old server back on in the Equinix data center so users can continue to access their email accounts. Then he spends a few days there setting up a new server. When he leaves, all the physical equipment for the new server is successfully installed except for an intrusion detection device, which Equinix installs later, once it gets shipped.

Meanwhile, the PRN employee who works remotely (Combetta) does his remote work to get the new server online. Around June 30, 2013, this employee begins to transfer all the email accounts from the old server to the new one. After several days, all email accounts hosted on the presidentclinton.com, wjcoffice.com, and clintonemail.com domains are transferred. However, PRN keeps the old server online at the Equinix data center along with the new server to ensure email continues to be delivered. But the old server no longer hosts email services for the Clintons.

According to an FBI report made public in September 2016, “The new Clinton email server hosted email for [Hillary] Clinton, President Clinton, [redacted], and their respective staffs.”

130601DellPowerEdgeR620

The Dell PowerEdge R620 (Credit: public domain)

This same FBI report will explain that the new server consists of the following equipment: “a Dell PowerEdge R620 server hosting four virtual machines, including four separate virtual machines for Microsoft Exchange email hosting, a BES for the management of BlackBerry devices, a domain controller to authenticate password requests, and an administrative server to manage the other three virtual machines, a Datto SfRlS 2000 to store onsite and remote backups of the server system, a CloudJacket device for intrusion prevention, two Dell switches, and two Fortinet Fortigate 80C firewalls.” (Federal Bureau of Investigation, 9/2/2016)

The FBI report will not make entirely clear what happens to the data on the old server. But a September 2015 Washington Post article will assert that after PRN moved all the data onto a new server, everything on the original server was deleted until it is “blank.” However, it was not wiped, which means having the old files overwritten several times with new data until they can never be recovered. (The Washington Post, 9/12/2015)

June 24, 2013—August 2015: Another company stores the contents of Clinton’s email server on a cloud storage system; this could help the FBI recover deleted emails.

The Datto, Inc. office in Rochester, New York. (Credit: The New York Times)

The Datto, Inc. office in Rochester, New York. (Credit: The New York Times)

Shortly after taking over management of Clinton’s private email server, Platte River Networks (PRN) buys a device called the Datto SIRIS S2000 from another company called Datto, Inc. that frequently makes copies of all the server’s contents. They use this device on a copy of Clinton’s server, which has been moved to a data storage facility in New Jersey. Then, apparently without PRN asking or paying for it, or even being aware of it, Datto stores those copies of the server’s contents on a “cloud” storage system elsewhere. (McClatchy Newspapers, 10/6/2015)

A September 2016 FBI report will explain, “At the Clintons’ request, PRN only intended that the backup device store local copies of the backups. However, in August 2015, Datto informed PRN that, due to a technical oversight, [Clinton’s] server was also backing up the server to Datto’s secure cloud storage. After this notification, PRN instructed Datto to discontinue the secure cloud backups.”

The FBI report will also reveal that the first Datto back-up takes place on June 24, 2013. But a new server is still being set-up and data being transferred from the old server, so the June 29, 2013 back-up will later prove most useful to FBI investigators. the FBI will say the back-ups will stop on December 23, 2013, but it isn’t explained why. (Federal Bureau of Investigation, 9/2/2016)

In 2015, an unnamed source familiar with Datto’s account will say that PRN was billed for “private cloud” storage, and since PRN didn’t have a cloud storage node of its own, the data bounced to Datto’s cloud. This source says that even though nobody seemed to realize it, Datto was “managing the off-site storage throughout.”

When asked if the FBI might recover Clinton’s deleted emails from Datto’s storage, the source will say, “People don’t use Datto’s service for getting rid of data.” Apparently, the FBI will ask for and get the contents of Datto’s storage in September 2015. (McClatchy Newspapers, 10/6/2015)

Senator Ron Johnson (R), who will write a letter to Datto in late 2015 seeking more information, will say that “questions still remain as to whether Datto actually transferred the data from its off-site data center to the on-site server, what data was backed up, and whether Datto wiped the data after it was transferred.” It is also unknown if Datto employees have security clearances allowing them to view classified information. (CNN, 10/8/2015) 

A Datto official will later say that investigators may be able to recover Clinton’s deleted emails if the data was on the server at the time Datto’s service was first used in 2013. (The Washington Post, 10/7/2015)

The FBI will later confirm that Datto back-ups to the cloud will occur, but it isn’t clear if the FBI recovered any emails from this that they didn’t find through other means. It also isn’t clear if the June 29, 2013 back-up that the FBI finds useful is from the Datto SIRIS S2000, the cloud, or both. (Federal Bureau of Investigation, 9/2/2016)

Also, despite internal PRN emails from August 2015 indicating many PRN employees didn’t know about the Datto cloud back-up until that time, by November 2015, the FBI will find evidence that an unknown PRN employee deleted data from the cloud back-up in March 2015.

Around Late March 2015: An Internet cloud back-up of Clinton’s server is deleted at this time, despite the company managing the server seemingly not knowing the cloud copy exists.

On November 19, 2015, an unnamed Datto executive will be interviewed by the FBI. Datto had provided back-up service and equipment to Platte Rivers Networks (PRN) when PRN was managing Clinton’s private server from June 2013 onwards. It will later be reported that in early August 2015, PRN employees discovered that in addition to a Datto back-up device attached to Clinton’s server, Datto had been also backing up Clinton’s server to the Internet “cloud.” Some internal PRN emails from early August 2015 show some employees acting surprised after being told about this.

A graphic of Datto's cloud structure. (Credit: Datto, Inc.)

A graphic of Datto’s cloud structure. (Credit: Datto, Inc.)

However, according to a later FBI summary of the Datto executive’s interview, he said that PRN must have known about the cloud back-up all along. “As evidence, [he] stated the partner portal, that PRN had log-in credentials to, had a feature displaying backed-up data an options to ‘delete cloud’ or ‘delete local.’ [He] stated PN would have seen their back-ups under ‘delete cloud.'”

More crucially, during the interview, the FBI will show him a Datto document “indicating email records were manually deleted from the Datto secure cloud back-ups of the [Clinton] server in March 2015.” He then will tell the FBI that it couldn’t have been a Datto employee who made the deletions, because there would have been a work ticket created showing that. Furthermore, IP addresses associated with the deletions indicate that someone from PRN must have done it, although PRN had a shared account so it can’t be proven who exactly made the deletions. (Federal Bureau of Investigation, 10/17/2016)

A Datto letter sent to the FBI in October 2015 will indicate that Datto technical experts reviewed administrative files and discovered through the device’s Internet interface that a series of deletions took place on the device on March 31, 2015, between 11:27 a.m. and 12:41 a.m. Furthermore, a much greater amount of data had been “deleted automatically based on the local device’s then-configured pruning parameters.” (US Congress, 9/12/2016) It is unclear if this refers to data deleted from the local Datto device or the Internet cloud back-up.

Although it is unknown who made these deletions, in a May 2016 FBI interview, PRN employee Paul Combetta will confess to deleting all of Clinton’s emails on her server as well as the Datto back-up device in precisely this time period, between March 25, 2015 and March 31, 2015.

October 23, 2015: A computer company tells the FBI that its back-up copy of Clinton’s private server data was deleted in late March 2015.

Steven Cash (Credit: LinkedIn)

Steven Cash (Credit: LinkedIn)

Steven Cash is a lawyer for Datto, Inc., the company that has been backing up the data on Clinton’s private server. They have been subcontracted to do this by Platte River Networks (PRN), the company managing the server. Cash emails an unnamed FBI agent, informing him of several issues to be aware of prior to a conference call planned for later that day.

A Datto hard drive, the Datto SIRIS S2000, has been attached to Clinton’s server since June 2013. Cash says that Datto technical experts have reviewed administrative files and discovered through the device’s Internet interface that a series of deletions took place on the device on March 31, 2015, between 11:27 a.m. and 12:41 a.m. The data had a date range from January 28, 2015 to March 24, 2015.

Furthermore, a much greater amount of data had been “deleted automatically based on the local device’s then-configured pruning parameters.” Cash writes that “These manual requests were requested from the Local Device’s web interface for the [redacted] agent…” (US Congress, 9/12/2016) While it is possible a person’s is in the redacted space, it could also be something such as “PRN employee.”

In a May 2016 FBI interview, PRN employee Paul Combetta will confess to deleting all of Clinton’s emails on her server as well as the Datto back-up device in precisely this time period, between March 25, 2015 and March 31, 2015. It is not known if the FBI knew of the deletions prior to this letter from Datto. However, the letter certainly makes it clear, but this will not become public knowledge until an FBI report released in September 2016, almost one year later.