Nvidia Acquires Mellanox to Accelerate Innovation; LightCounting Comments on Implications of the Deal for Optical Interconnect Technologies
The bottom line is that the deal will accelerate innovation and optical interconnects will be one of the areas impacted.
The deal emphasizes that combination of GPUs (Graphical Processing Units, made by Nvidia) with low latency switching and broadband interconnects (made by Mellanox) is the key for high performance computers (HPCs) and datacenter clusters running AI and machine learning applications. It is entirely possible that the inspiration for the Mellanox acquisition first began as Nvidia’s best engineers recently worked alongside Mellanox’s on both the reigning number 1 supercomputer Summit and the current number 2 supercomputer Sierra each of which use a proprietary interface called NVLink between the IBM Power9 CPUs and the Nvidia Volta GPUs at each node and high-speed Dual-Rail Mellanox 100G EDR InfiniBand connections between the individual CPU/GPU compute nodes. There are 4,608 CPU/GPU compute nodes in Summit and 4,474 such nodes in Sierra. Summit “only” fills up the space of two tennis courts but contains 185 miles of fiber optic connections.
In 2018, 88% of the Top 500 supercomputers were clusters - a loosely coupled collection of servers, similar to a data center (www.top500.org). An alternative is Massively Parallel Processing (MPP) systems with ‘tightly coupled’ processor architecture. InfiniBand connectivity (developed by Mellanox) and Omni-Path connections (developed by Intel) are used in 35% of the Top 500 systems built as clusters. InfiniBand, built by Mellanox, connects 55% of the Top 500 machines that are real supercomputers rather than cloud clusters. Many of these connections use active optical cables, discussed in the AOC/EOM report, published by LightCounting in December 2018.
The high-speed HPC cluster segment uses the InfiniBand and Omni-Path Architecture protocols due to their very low latency and low overhead as the protocol stack is much simpler and smaller than Ethernet’s. HPC machines are traditionally thought of as being used for scientific applications such as weather simulation or global warming but a large percentage of the shipments are moving into main stream engineering applications within corporations, Web 2.0 applications such as for Hadoop storage acceleration, and in cloud computing applications where the HPC services are rented. More recently, artificial intelligence (AI) and machine learning applications have become a very key application within HPC and within data centers. Most of these applications involve huge amounts of compute and massive data movement. In some cases such as the financial markets where “flash” stock trading systems trade billions of stock trades in milliseconds – low latency is the prime requirement.
Machine learning is the ‘training’ side of artificial intelligence. It can operate on massive data sets and the use of GPUs (Graphical Processing Units) such as NVIDIA Volta accelerators [called tensor processors, shown in Figure 1] is now typical. Google has built their own ASIC in this category. Facebook built a machine learning system that ranks at #50 on the Top500 list and uses NVIDIA DGX-1 machines populated with their GPUs and connected with InfiniBand.
Future generations of machine learning clusters are likely to take advantage of a disaggregated design, illustrated in Figure 2. Instead of combining a limited number of CPUs, GPUs and memory in standard servers comprising a cluster, all the cluster resources can be accessed and interconnected to create an optimal combination of CPUs, GPUs and memory for a specific task.
Emergence of disaggregated systems is very important for optical connectivity because they require 10x-100x more bandwidth. Also, these systems may benefit from optical switching or optical bandwidth steering technologies, according to Prof. Keren Bergman at Columbia University. This topics was addressed in a special session at OFC last week.
With the acquisition of Mellanox, Nvidia has access to all the key technologies for implementing disaggregated clusters. It is a pity that Mellanox was forced by its activist investors to shut down their development of Silicon Photonics technologies last year. Nvidia may restart these activities or acquire another company.
Cisco’s acquisition of Luxtera – a silicon photonics pioneer in the end of 2018 may spark a few more deals this year. Commenting on Luxtera’s deal at OIDA Executive forum at OFC, Bill Garner of Cisco, acknowledged that the primary reason for having silicon photonics technology in house is developing of next generation ASICs with co-packaged optics. Intel and Xilinx plan to have optics co-packaged with FPGAs as early as next year. Optical connectivity co-packaged with GPUs must be on Nvidia’s agenda and having this technology in house is certainly a plus. Adding Mellanox low latency switching technology to the mix makes it much more likely that optical switching will find an application in datacenter clusters running AI applications. Nvidia may be the first company to find the right way to do this and accelerate innovation along the way.
About LightCounting Market Research
LightCounting -- The name alone is what sets us apart and defines us as a company. We are a leading optical communications market research company, offering semi-annual market updates, forecasts, and state-of-the-industry reports based on analysis of primary research with dozens of leading optics component, module, and system vendors, as well as service providers and cloud companies. LightCounting is the optical communications market's source for accurate, detailed and relevant information necessary for doing business in today's highly competitive environment. Register to receive our monthly newsletter: LightCounting.com or connect with us on LinkedIn and Twitter.
Interested in meeting with LightCounting at these upcoming industry events? Email us today to schedule a meeting with our team. View our recently published reports and 2019 Research Roadmap.
3D Sensing for Self-Driving Cars Reaches the Peak of Inflated Expectations
LightCounting releases a new report addressing illumination in smartphones and automotive lidarIn 2019, the market for VCSEL (vertical cavity surface-emitting laser) illumination in smartphones will exceed $1.0 billion – now nearly triple the size of the market for communications VCSELs. That’s quite remarkable for a market that didn’t exist three years ago.3D sensing in smartphones felt like an overnight sensation, but the technology foundations were laid down years ago with Microsoft’s Kinect – a motion-sensing peripheral for gamers released in 2010 but discontinued in 2017 after lackluster sales. Lumentum supplied lasers to the Kinect almost a decade before the iPhone opportunity emerged; the company was ready to profit from the iPhone X opportunity when Apple decided to launch 3D sensing for facial recognition in September 2017.
Figure: 3D depth-sensing meets the Gartner Hype Cycle
Source: Gartner with edits by LightCounting
If all technologies follow the Gartner Hype Cycle, shown in the Figure above, then 3D sensing in smartphones is now moving up the slope of enlightenment. Android brands raced to add 3D sensing to their flagship phones in 2018 – the Xiaomi Mi8 Explorer and Oppo Find X phones were first – although these only sold in single digit million quantities. Huawei also brought out new phones with 3D sensing, but the ongoing U.S. export ban on the Chinese company must be hurting the company’s traction outside China. Apple continues to dominate the market as all new iPhones released by Apple since 2017 have included 3D sensing on the front of the phone. Apple is expected to introduce 3D sensing for ‘world-facing’ applications in 2020, which adds another laser chip to every phone.
Last year illumination for lidars were not included in our market forecast since LightCounting considered it unlikely that lidar would penetrate the consumer market to any great extent over the forecast period. All indicators now point to a market for lidar illumination ramping up in 2022 and beyond. Optical components firms are now shipping prototypes and samples of VCSELs, edge emitters and coherent lasers to customers developing next-generation lidar systems – many of them building on their expertise in illumination for optical communications and smartphones.
As was the case with smartphones, the foundations for lidar technology were laid down much earlier – in this case with the DARPA Challenge 2007, where the winning vehicle used a 64-laser lidar system from Velodyne Acoustics (now Velodyne Lidar). Lidar is considered by the majority of the industry to be an essential part of the sensor suite required for autonomous driving, helping the vehicle to navigate through the environment and detect obstacles in its path. The first commercial deployments have begun. In Germany, lidar on the Audi A8 enables the car to drive itself for limited periods under specific conditions. In Phoenix, Arizona, you can hail a ride in a Waymo robotaxi.
Investor enthusiasm for lidar is undeniable with nearly half a billion dollars invested in lidar start-ups in 2019 according to our analysis of publicly available investment data. Notable deals include $60 million for U.S. company Ouster in March, Israel’s Innoviz Technologies Series C round of $132 million in the same month, and $100 million for U.S.-based Luminar Technologies in July. Interestingly, these examples illustrate the variety of lidar approaches: each company is building a different type of lidar based on a different wavelength: 850nm for Ouster, 905nm for Innoviz and 1550nm in the case of Luminar. There’s an open technology battle and they can’t all be winners.
The automotive lidar market seems to be close to the peak of ‘inflated expectations’. It’s easy to understand why. The automotive industry is enormous, with nearly 100 million vehicles (including trucks) produced annually. Players like Baidu, GM Cruise and Waymo are backed by deep corporate pockets, and new entrants like Aurora and Pony.ai are attracting hundreds of millions in investment. Intel’s $15.3 billion purchase of Mobileye in 2017 was also directed at autonomous driving. Sensor company AMS is in a $4.8 billion battle to acquire German semiconductor lighting firm Osram with its eye firmly on lidar.
However, signs indicate that the descent into the trough of disillusionment could have already begun. Waymo has yet to roll out its robotaxi services more widely – and this summer admitted that its vehicles needed more testing in the rain. GM Cruise has delayed launch of commercial services for self-driving cars beyond 2019 and is reluctant to commit to a new timescale, with its CEO Dan Ammann observing that safety is paramount; automotive is not an industry where you can “move fast and break things” he said. A casualty of the slow pace was optical phased array lidar developer Oryx Vision, which closed its doors in August and started to hand money back to investors.
While lidar is being deployed commercially today, prices are not conducive to mass production, and there are open questions around regulation, safety, ethics and consumer acceptance. Do local laws prohibit self-driving cars? Will they really be safer than humans? Who is responsible for a crash? LightCounting remains skeptical about the pace of adoption of autonomous vehicles, but will be watching the market closely and with optimism.
More information on the report is available at: https://www.lightcounting.com/Sensing.cfm.