ThousandEyes: the X-ray machine of the internet

It’s the early 2010s, and Mohit Lad and Ricardo Oliviera are working well into the evening, developing their internet monitoring software ThousandEyes in their startup’s first office in San Francisco. The city is energy conscious enough that the lights in the building will go off at 6pm on the dot, and it takes a phone call and a passcode to get things back up and running. Oliviera has had enough of this, and has written a script using Twilio, which offers APIs to automate phone calls.

This works for a week, until the lights turn off of their own accord again. After frantically debugging the script in the dark, the founders realize that their script is absolutely fine. The problem is that Twilio is hosted on an Amazon Web Services (AWS) data center on the other side of the country, which has been brought down due to a storm.

As he speaks from Austin, Texas in 2022, Lad thinks this was a prescient moment to describe the way the internet works today.

The changing internet

“Every time there’s an Amazon outage, something breaks because the way applications are being built right now, there’s a lot more API calls than ever before,” he tells TechRadar Pro. 

“Previously, you would see – 10 years ago, 20 years ago – when you were building applications, you would include the code inside through libraries. Now you do an API call. An API call means you insert a dependency into some provider that may be sitting somewhere you don’t know. 

“So as things get concentrated, if there are outages in parts of Amazon’s environment, what happens is even things you don’t anticipate breaking will break, like your doorbell cam may not work because they have an API call where on Amazon.” 

“And I think one, one pattern you will start to find is that there’s more and more unpredictability that will come through in terms of ripple effects. When large networks or large hosting providers, cloud providers go down.”

The best illustration for how the internet has changed, according to Lad, is the switch from data being stored on a business’ own premises to trusting cloud service providers, like AWS, Google Drive, and Microsoft Azure, with the data, usually as a cost-cutting measure. That, and a familiar interface, are the most obvious benefits, but this too, comes at a price.

“Companies used to put everything in their own data center,” he notes. “Now they’re going into cloud, they don’t control it. They used to build their applications in their own premises, like a CRM, or HR application. And even that is now done on Salesforce, Workday, or Office 365. We’re using Teams, right? Teams is hosted in the cloud.” 

“The single thing that connects all of this together is the internet. And if it doesn’t work. Or portions of it don’t work, then it severely impacts user experience. The whole concept of ThousandEyes was started because we believe that the quality of the internet impacts quality of life.” 

The ThousandEyes software

Part of what makes ThousandEyes indispensable to over 170 Fortune 500 companies, the top ten banks in the US, and customers such as Mastercard, Volvo, and HP, is that it maps routes between vital company infrastructure and the cloud providers hosting it.

“Think about Google Maps, or Waze. It’s all about providing a visual around what’s happening between point A and point B, so you can make the right decision,” says Lad. “That sort of end-to-end view of what the journey is between your end users and application, which is missing in the current market world.”

Lad maintains that ThousandEyes remains a vital resource because of the way the internet works. “The internet is essentially a collection of different networks. What ThousandEyes is doing is providing a view showing that journey and highlighting if there’s an outage somewhere, and that gives you the ability to route around it.”

To illustrate, he shares what he calls “the 30,000 feet view” inside the ThousandEyes software – a total overview of outages across the internet, with the estimated geographical impact depicted on an interactive map.

The closest, publicly available approximation to the “30,000 feet view” Lad showcased, available on the ThousandEyes website. (Image credit: ThousandEyes)

Another example Lad gives of ThousandEyes’ unique functionality is its ability to drill down into where exactly an outage is coming from within a network. He picks an ongoing outage at random – a US provider is down, impacting traffic coming from Australia, via Cloudflare.

“So if we drill down, it shows up as Sydney, and you can look at specific parts of this network in Sydney, where the outages are,” he explains. “And knowing this, if you’re using this provider and you have critical customers in Australia, you actually know there’s an outage going on in that part of the environment. You can avoid this network, and make sure your users have a consistent experience and be able to help them out of a blind spot.”

Getting ThousandEyes funded

Since the company began around ten years ago, Lad says that this belief has only become more important. The road to getting to where he and ThousandEyes are today was difficult, and, it turns out, a last ditch effort. Lad’s plans to enter into academia at the University of California, Los Angeles (UCLA) were scuppered, and his US visa was running out. Something had to be done.

In July 2008, Lad began working at an internet startup, but within two months was told that he was being let go as a result of layoffs because of the financial crisis. This, he says, opened his eyes. Oliviera, his labmate at UCLA, had been asking him to start a company with him, and he wasn’t interested. Redundancy, Lad says, made him decide to take a chance, although it initially seemed an unlikely one.

“Nobody was willing to fund ThousandEyes,” he notes. “A lot of people didn’t think the internet was going to be important enough to be monitored. We ended up raising money from the National Science Foundation, from the US government.”

That initial grant of $150,000 built the first version of ThousandEyes, but Lad is quick to point out that, in practice, it wasn’t a life-changing sum of money. ThousandEyes’ first datacenter, he says, was built in a garage with servers that had been thrown out into the street.”

“One of the things that came out of [the financial crisis] was a lot of equipment being put in recycle bins outside companies in the Bay Area. Even today, we have the first server we put in place in our office.”

Ricardo Oliveira (left) and Mohit Lad (right) with ThousandEyes’ first server, still live and kicking. (Image credit: ThousandEyes)

The state of funding for technology startups right now isn’t quite so bleak, says Lad. “Last year was quite crazy. Everybody was throwing a lot of money. I think, this year, investors are more disciplined around fundamentals, and are being pretty selective about where they invest and how they invest. And sometimes these market shifts are a good opportunity to really understand where to focus.” 

“Take ThousandEyes as an example. If we had gotten a lot of money from day one, we would have gone and tried all these different things to build a product and probably failed. The fact that we had very little money meant that we had to really focus on the one thing we could sell. And sometimes I feel like overfunded companies are essentially writing their own failure when they raise too much money and try to grow faster.”

“There’s still a lot of investor money. […] There are other government programmes in different countries, and I would definitely encourage people to leverage [them]. Sometimes these programmes won’t give you quick money – [the National Science Foundation grant was] spread out over increments every three months. But it does help you, and puts more discipline in how you operate. So, I think: look for alternative ways.”

“My recommendation to entrepreneurs building software companies [today] is to focus on getting early customers. That’s the best way to build the company, validate the product.”

Predicting the internet

To commit to monitoring the internet is also to commit to the idea that the internet is constantly changing. It’s the first thing Lad admits when we ask him about the unpredictability of ThousandEyes’ monitoring solutions. That change can be problematic, he says, in that companies’ own senses of self-preservation sometimes govern the internet.

“[The internet’s] also very complex, and it’s not controlled by one entity. So every network is making decisions that are sometimes in their own self-interests. For example, occasionally networks will announce IP addresses that don’t belong to them, and they can suck traffic into their site, and this is how even big sites like Google have gone down. This is what’s called BGP hijacks, or route hijacking. ” 

“Even the best networks with the best engineers can’t control availability, because somebody else on the internet announces that they are Google and traffic starts going to them. This is part of what makes the internet really fascinating, and really difficult to predict.”

The ThousandEyes platform in action. (Image credit: ThousandEyes)

ThousandEyes is by no means giving up hope, though, and Lad says that the company has built technology that uses historical data to predict outages at particular times of the day – much like those early days for the company in San Francisco, but without having to wait for the outages to happen before companies can react.

“We can’t predict every outage,” he’s quick to point out. “We won’t predict if a lightning strike takes out a data center. But if there’s a certain pattern that we can predict based on past data – for example, at 9AM on a Tuesday you always have issues with Microsoft 365 from this office, but you won’t have this problem if you switch [internet service providers]. That’s the kind of prediction we can make.”

The mention of natural disasters is genuinely surprising, and we have to admit that we haven’t thought too much about how environmental factors, or indeed climate change, are impacting the stability of the internet. Separately, Lad uses recurring fires in New York, which impact a large portion of the East Coast’s ability to connect to the internet, and beavers chewing through cables as examples of unpredictable events that affect connectivity.

Cisco’s acquisition of ThousandEyes

Lad claims that a large part of ThousandEyes’ ability to help as many companies as possible (and, by extension, people) is the company’s acquisition by Cisco in 2020 which, as it turns out, is an unusual and engaging story.

“Cisco was a customer of ThousandEyes from 2014, and they actually spoke at our conferences as well,” he explains. “They were using it on their internal side to understand their employees’ experiences, as they were adopting the internet more and more between their offices, moving things to cloud, so they were using ThousandEyes internally to be proactive on outages that were happening.”

“Every product functionality we released, they were adopting it. And I think a couple of things happened. One: the Cisco engineering teams made some changes where they decided that the Cisco platform should potentially run other applications as well, and when they were making these changes, the IT team, the customer side of Cisco, came to us and wanted to run ThousandEyes on all the Cisco devices in the branch offices. So we worked with them to get 1000s running on the Cisco devices. 

“A couple things happened as a result [of that]. The leadership side [of Cisco], all the way to the CEO, started learning about how ThousandEyes [was] being used within Cisco. And Cisco being the company that basically helped build the internet, [they think] “if there’s something that we’re using as a first party, we should look at it.”

“The other thing [was] that, before we came to Cisco, we had more than a hundred Fortune 500 [customers]. So every time Cisco would have advisory meetings with some of these customers, they would bring ThousandEyes up to the leadership and say “you should consider partnering.”

Lad believed – and still believes – that Cisco and ThousandEyes’ goals were aligned: wanting to build the best view of the internet. He and Oliviera thought that, by partnering with Cisco, they would be best equipped to gather data from all corners of the world. 

Lad has agreed to this interview largely to commemorate the two-year anniversary of the acquisition, although it is still tempting to ask him whether he finds it difficult being a huge cog in a big machine. However, he preempts this question before it’s asked, by discussing ThousandEyes’ unique role within Cisco.

“We’re set up as an independent business unit, not part of a certain product group within Cisco. And that’s because the internet threads to every piece of Cisco’s business. And because we help monitor, visualize, and understand the internet, [we’re] helping all the product lines sort of benefit from the galvanized data. […] We’re still operating like a startup, except at that Cisco scale.”

He says that one of the key benefits of the Cisco partnership is the ability to get first meetings with customers, as the ThousandEyes company can trade on Cisco’s brand recognition. Another is that, by continuing to integrate ThousandEyes with Cisco devices, the amount of data that ThousandEyes can collect is increasing exponentially. “I look back,” he says, “and it was one of the best decisions we made.”

The internet in the future

As Lad looks forward, he thinks that there are challenges coming, but, equally, to celebrate.

“There isn’t one thing I would say that’s going to change,” he says. “I just feel connectivity is so critical that people live their lives around just being able to connect to something really quickly.” 

“It’s also the devices that are becoming increasingly internet connected. That’s also going to challenge how the internet is evolving and how it needs to support all these billions of billions of devices that are coming online.”

“The last thing I would add is that there’s a large population of the world that is still not online. And there are areas, especially in Africa, and parts of Asia and India, where people are connected through their cell phones. […] And I do think, in those markets in particular, the ecosystem is evolving around that lifecycle versus building things for desktops and laptops.”

Lad isn’t necessarily convinced that future changes to the way humanity takes to technology will be quite so rapid. “It’s not typical to have such a rapid acceleration of a trend overnight, so I feel like some of these changes are going to be accelerated more regionally, or are going to be gradual, globally. But [things] will continue to evolve.”

“What I see is that some of the changes are stirred by an application that makes life easier for people. And suddenly people start coming online or using a cell phone because they can get something done. They can accept payments for their work [for example], so that’s the reason why they have a cell phone.”

For the most part, Lad won’t be drawn to discuss fears about the future of the internet. He describes himself as an optimist for the internet’s future, and continues to see it as a force for good. There’s an overriding sense that to him, ThousandEyes is not about keeping watch over and pushing back against the internet, but embracing it as a tool, and working to keep it accessible.

“Obviously, the internet is one of the reasons why your online accounts can get hacked, and all of that. So there’s definitely a lot of bad stuff happening, but I think my hope and optimism is the good trumps the bad.”

“And we’re able to use it for the right reasons. And it really continues to change lives and connect people and create, you know, an experience where people from different parts of the world are able to come together no matter where they are.”