I was interested in what someone from such a large website, eBay, would have to say about its architecture. There was a lot of takeaways from it, summarized in four main ideas: partition everything, use asynchrony everywhere, automate everything, and design the system keeping in mind that everything fails at some pint in a large distributed system. Although this is an older talk (from 2007), these all seem to be sound systems to design something so large. He referenced a theorem from 1998 that he said took a while to get traction. Likewise, I am sure his design principles have proven true for the last ten years.
To be completely honest, there was a lot of industry-specific jargon used that went above my head. I took a lot of notes though, and looked up the terms I did not know. I still found it very interesting, though. I found it the most interesting that someone designing one of the largest websites in the world was incredibility realistic of the shortcomings of everything designed. It seemed that everything designed had to have a degree of compromise of what was expected of it. He referenced the “CAP Theorem,” where it seemed you cannot always have all three parts that it stood for — “Consistency,” “Availability,” and “Partition tolerance.” He also was realistic that everything can fail, especially when the system gets as big as it gets bigger. For some reason it surprised me when someone from such a prominent company would admit this.
Something else that really surprised me what he said one of their biggest limiting factor was something I would not have expected, which was power consumption. He claimed that some companies’ database centers can use more energy than a local power station can support. This makes it much more important to design efficient database structures, especially in a company that deals with data in several petabytes.
It just boggles my mind how reliable websites like this are for millions upon millions of consumers. He said $2,000 worth of transactions were made every second, and that was over ten years ago. The site has grown since then. This gave me some great ideas to run with, as I hope to one day work for a company as esteemed as his.
Link: http://www.se-radio.net/2008/09/episode-109-ebays-architecture-principles-with-randy-shoup/
To be completely honest, there was a lot of industry-specific jargon used that went above my head. I took a lot of notes though, and looked up the terms I did not know. I still found it very interesting, though. I found it the most interesting that someone designing one of the largest websites in the world was incredibility realistic of the shortcomings of everything designed. It seemed that everything designed had to have a degree of compromise of what was expected of it. He referenced the “CAP Theorem,” where it seemed you cannot always have all three parts that it stood for — “Consistency,” “Availability,” and “Partition tolerance.” He also was realistic that everything can fail, especially when the system gets as big as it gets bigger. For some reason it surprised me when someone from such a prominent company would admit this.
Something else that really surprised me what he said one of their biggest limiting factor was something I would not have expected, which was power consumption. He claimed that some companies’ database centers can use more energy than a local power station can support. This makes it much more important to design efficient database structures, especially in a company that deals with data in several petabytes.
It just boggles my mind how reliable websites like this are for millions upon millions of consumers. He said $2,000 worth of transactions were made every second, and that was over ten years ago. The site has grown since then. This gave me some great ideas to run with, as I hope to one day work for a company as esteemed as his.
Link: http://www.se-radio.net/2008/09/episode-109-ebays-architecture-principles-with-randy-shoup/
Comments