Author: Kristián Ivančo

0:00 / 0:00

Back in 2019, the Orion team prepared an important demo for Meta Founder & CEO Mark Zuckerberg, showcasing potential waveguides for augmented reality glasses—a pivotal moment when theoretical calculations on paper were brought to life. And it was a demo that changed everything.

“Wearing the glasses with glass-based waveguides and multiple plates, it felt like you were in a disco,” recalls Optical Scientist Pasqual Rivera. “There were rainbows everywhere, and it was so distracting—you weren’t even looking at the AR content. Then, you put on the glasses with silicon carbide waveguides, and it was like you were at the symphony listening to a quiet, classical piece. You could actually pay attention to the full experience of what we were building. It was a total game changer.”

Yet as clear (pun intended) as the choice of silicon carbide as a substrate seems today, when we first started down the road to AR glasses a decade ago, it was anything but.

“Silicon carbide is normally heavily nitrogen-doped,” says Rivera. “It’s green, and if it gets thick enough, it looks black. There’s no way you could make an optical lens out of it. It’s an electronic material. There’s a reason it’s that color, and it’s because of electronic properties.”

“Silicon carbide has been around as a material for a long time,” agrees AR Waveguides Tech Lead Giuseppe Calafiore. “Its main application is high-power electronics. Take electric vehicles: All EVs require a chip—but that chip must also be capable of very high power, moving the wheels, and driving this thing. It turns out you can’t do it with the regular silicon substrate, which is what makes the chips that we use in our computers and electronics. You need a platform that allows you to go through high currents, high power, and that material is silicon carbide.”

Until fairly recent discussions around renewable energy sources started to heat up, the market for these high-power chipsets was nowhere near the size of the market for chips for consumer electronics. Silicon carbide has always been expensive, and there wasn’t much incentive to bring costs down because, for the size of chip that you make for a car, the price of the substrate was tolerable.

“But it turns out that silicon carbide also has some of the properties that we need for waveguides and optics,” Calafiore says. “The refractive index is the key property that we care about. And silicon carbide has a high refractive index, which means that it’s capable of channeling in and outputting a large quantity of optical data. You can think of it as optical bandwidth—just like you have bandwidth for the internet, and you want it to be large enough so that you can send huge amounts of data through that channel. The same goes for optical devices.”

The higher a material’s refractive index, the higher its étendue, so you can send more optical data through that channel.

“The channel in our case is our waveguide, and a larger étendue translates to a larger field of view,” explains Calafiore. “The larger the refractive index of a material, the larger field of view the display can support.”

The Road to the Right Refractive Index

When Calafiore first joined what was then Oculus Research in 2016, the highest refractive index glass the team had at its disposal was 1.8—which required stacking multiple plates to achieve the desired field of view. Undesirable optical artifacts aside, the assembly line became increasingly complicated as the first two waveguides had to be perfectly aligned, and then that stack had to be perfectly aligned with a third waveguide.

“Not only was it expensive, it was immediately obvious that you couldn’t have three pieces of glass per lens in a pair of glasses,” Calafiore recalls. “They were too heavy, and the thickness was prohibitively large and ugly—nobody would buy it. So we went back to square one: trying to boost the refractive index of the substrate to reduce the number of plates needed.”

The first material the team looked into was lithium niobate, which has a refractive index of roughly 2.3—quite a bit higher than the glass at 1.8.

“We realized that we only had to stack two plates and could maybe even manage with one plate to still cover the field of view,” says Calafiore. “Almost in parallel, we started looking at other materials—which is how we figured out alongside our suppliers in 2019 that silicon carbide, in its purest form, is actually very transparent. It also happens to have the highest refractive index known for an optical application, which is 2.7.”

That’s a 17.4% increase over lithium niobate and a 50% bump over glass, for those keeping score at home.

“With a few modifications to the very same equipment that was used in the industry already, it was possible to get transparent silicon carbide,” Calafiore says. “You could just change the process, be a lot more careful, and instead of optimizing for electronic properties, you optimize for optical properties: transparency, uniformity of the refractive index, etc.”

The Potential Cost of Compromise

At that time, the Reality Labs team was the first to even attempt moving from opaque silicon carbide wafers to transparent ones. And because silicon carbide is one of the hardest known materials, it essentially requires diamond tooling to cut or polish it. As a result, the non-recurring engineering costs were very high, so the resulting substrate was quite expensive.

While more cost-effective alternatives exist, as with any technology, they each have tradeoffs. And as the field of view increases towards Orion’s industry-leading field of view of approximately 70 degrees, new issues arise like ghost images and rainbows.

“Finding the optimum solution for a wide field of view AR display is fraught with tradeoffs between performance and costs,” explains Director of Research Science Barry Silverstein. “Costs can often be driven down, but if performance doesn’t cut it, costs won’t ultimately matter.”

Ghost images are like visual echoes of the primary image being projected onto the display. Rainbows are colorful streaks of light that are created when ambient light reflects off of the waveguide. “Say you’re driving at night with moving car lights around you,” says Silverstein. “You’re going to have rainbows that move, too. Or if you’re out on the beach playing volleyball and the sun’s shining, you’re going to get a rainbow streak that’s moving with you and you’ll miss your shot. And one of the miraculous properties of silicon carbide is that it gets rid of those rainbows.”

“The other advantage of silicon carbide that none of the other materials have is thermal conductivity,” Calafiore adds. “Plastic is a terrible insulator. Glass, lithium niobate, same thing. You go to silicon carbide and it’s transparent, looks like glass, and guess what: It conducts heat.”

So in July 2020, the team determined that silicon carbide was the optimum choice for three primary reasons: It led to an improved form factor because it required only a single plate and smaller mounting structures, it had better optical properties, and it was lighter than dual-plate glass.

The Secret of Slant Etch

With the material in mind, the next nut to crack was the fabrication of the waveguides—and specifically, an unconventional grating technique called slant etch.

“The grating is the nanostructure that in-couples and out-couples the light from the lens,” explains Calafiore. “And for silicon carbide to work, the grating needs to be slant etch. Instead of being vertical, you want the lines of the grating to be sloped diagonally.”

“We were the first ones to do slant etch directly on the devices,” says Research Manager Nihar Mohanty. “The whole industry used to rely on nano imprint, which doesn’t work for substrates with such a high refractive index. That’s why no one else in the world had thought about doing silicon carbide.”

But because slant etch is an immature technology, most semiconductor chip suppliers and factories lack the necessary tools.

“Back in 2019, my manager at the time, Matt Colburn, and I established our own facility as there was nothing in the world that could produce etched silicon carbide waveguides and where we could prove out the technology beyond the labscale,” explains Mohanty. “It was a huge investment, and we established the whole pipeline there. The tooling was custom-made for us by our partners, and the process was developed in-house in Meta, though our systems are research-grade because there were no manufacturing-grade systems out there. We worked with a manufacturing partner to develop manufacturing-grade slant etch tooling and processes. And now that we’ve shown what’s possible with silicon carbide, we want others in the industry to start making tools of their own.”

The more companies that invest in optical-grade silicon carbide and develop equipment, the stronger the category of consumer AR glasses will become.

No Longer Chasing Rainbows

While technological inevitability is a myth, the stars certainly seem to be aligning in silicon carbide’s favor. And although the team continues to investigate alternatives, there’s a strong sense that the right people have come together at the right time under the right market conditions to build AR glasses using this material.

“Orion proved that silicon carbide is a viable option for AR glasses,” says Silverstein, “and we’re now seeing interest across the supply chain on three different continents where they’re heavily pursuing this as an opportunity. Silicon carbide will come out on top. It’s just a matter of time in my book.”

And a lot can happen in that time—much like the way the tables have turned since we grew our first clear silicon carbide crystals.

“All of these silicon carbide manufacturers had cranked up the supply massively in response to the expected EV boom,” notes Calafiore. “Right now, there’s an overcapacity that didn’t exist when we were building Orion. So now, because supply is high and demand is low, the cost of the substrate has started to come down.”

“Suppliers are very excited by the new opportunity of manufacturing optical-grade silicon carbide—after all, each waveguide lens represents a large amount of material relative to an electronic chip, and all of their existing capabilities apply to this new space,” adds Silverstein. “Filling your factory is essential, and scaling your factory is the dream. The size of the wafer matters, too: The bigger the wafer, the lower the cost—but the complexity of the process also goes up. That said, we’ve seen suppliers move from four-inch to eight-inch wafers, and some are working on precursors to 12-inch wafers, which would yield exponentially more pairs of AR glasses.”

These advancements should help continue to drive costs down. It’s still early days, but the future is coming into focus.

“At the beginning of any new technological revolution, you try a bunch of things,” says Calafiore. “Look at television: We started with cathode-ray tubes, and then we went to the LED plasma TVs and now microLEDs. We went through several different technologies and architectures. As you pathfind, lots of paths end up nowhere, but there are some that you just keep coming back to as the most promising. We’re not at the end of the road, and we can’t do it alone, but silicon carbide is a wonder material that is well worth the investment.”

“The world is awake now,” adds Silverstein. “We’ve successfully shown that silicon carbide can flex across electronics and photonics. It’s a material that could have future applications in quantum computing. And we’re seeing signs that it’s possible to significantly reduce the cost. There’s a lot of work left to be done, but the potential upside here is huge.”

---

Learn more about silicon carbide in Photonics Spectra.

For more information on Orion, check out these blog posts:

• Zero to One: How Our Custom Silicon & Chips Are Revolutionizing AR
• Orion’s Compute Puck: The Story Behind the Device that Helped Make Our AR Glasses Possible

TL;DR:

• Today at Connect, Mark Zuckerberg unveiled Orion—our first pair of true AR glasses, previously codenamed Project Nazare.
• With an industry-leading field of view, silicon carbide lenses, intricate waveguides, uLED projectors, and more, Orion is our most advanced and most polished product prototype to date.
• Now, we’re continuing product optimizations and lowering costs as we drive toward a scalable consumer device that will revolutionize the way people interact with the world.

---

“We are building AR glasses.”

Five simple words, spoken five years ago. With them, we planted a flag on our vision of a future where we no longer have to make the false choice between a world of information at our fingertips and the physical world all around us.

And now, five years later, another five words that stand to once again change the game:

We have built AR glasses.

https://youtube.com/watch?v=2CJsnyS8u3c%3Fsi%3DTr77gOKEq3PnA7-k%26controls%3D0%3Fautoplay%3D0%26color%3Dwhite%26cc_lang_pref%3Den%26cc_load_policy%3D0%26enablejsapi%3D1%26frameborder%3D0%26hl%3Den%26rel%3D0%26origin%3Dhttps%253A%252F%252Fwww.meta.com

At Meta, our mission is simple: give people the power to build community and bring the world closer together. And at Reality Labs, we build tools that help people feel connected anytime, anywhere. That’s why we’re working to build the next computing platform that puts people at the center so they can be more present, connected, and empowered in the world.

Ray-Ban Meta glasses have demonstrated the power of giving people hands-free access to key parts of their digital lives from their physical ones. We can talk to a smart AI assistant, connect with friends, and capture the moments that matter—all without ever having to pull out a phone. These stylish glasses fit seamlessly into our everyday lives, and people absolutely love them.

Yet while Ray-Ban Meta opened up an entirely new category of display-less glasses super-charged by AI, the XR industry has long dreamt of true AR glasses—a product that combines the benefits of a large holographic display and personalized AI assistance in a comfortable, stylish, all-day wearable form factor.

And today, we’ve pushed that dream closer to reality with the unveiling of Orion, which we believe is the most advanced pair of AR glasses ever made. In fact, it might be the most challenging consumer electronics device produced since the smartphone. Orion is the result of breakthrough inventions in virtually every field of modern computing—building on the work we’ve been doing at Reality Labs for the last decade. It’s packed with entirely new technologies, including the most advanced AR display ever assembled and custom silicon that enables powerful AR experiences to run on a pair of glasses using a fraction of the power and weight of an MR headset.

Orion’s input system seamlessly combines voice, eye gaze, and hand tracking with an EMG wristband that lets you swipe, click, and scroll while keeping your arm resting comfortably by your side, letting you stay present in the world and with the people around you as you interact with rich digital content.

Beginning today at Connect and continuing throughout the year, we’re opening up access to our Orion product prototype for Meta employees and select, external audiences so our development team can learn, iterate, and build towards our consumer AR glasses product line, which we plan to begin shipping in the near future.

Why AR Glasses?

There are three primary reasons why AR glasses are key to unlocking the next great leap in human-oriented computing.

1. They enable digital experiences that are unconstrained by the limits of a smartphone screen. With large holographic displays, you can use the physical world as your canvas, placing 2D and 3D content and experiences anywhere you want.
2. They seamlessly integrate contextual AI that can sense and understand the world around you in order to anticipate and proactively address your needs.
3. They’re lightweight and great for both indoor and outdoor use—and they let people see each other’s real face, real eyes, and real expressions.

That’s the north star our industry has been building towards: a product combining the convenience and immediacy of wearables with a large display, high-bandwidth input, and contextualized AI in a form factor that people feel comfortable wearing in their daily lives.

Compact Form Factor, Compounded Challenges

For years, we’ve faced a false choice—either virtual and mixed reality headsets that enable deep, immersive experiences in a bulky form factor or glasses that are ideal for all-day use but don’t offer rich visual apps and experiences given the lack of a large display and corresponding compute power.

But we want it all, without compromises. For years, we’ve been hard at work to take the incredible spatial experiences afforded by VR and MR headsets and miniaturizing the technology necessary to deliver those experiences into a pair of lightweight, stylish glasses. Nailing the form factor, delivering holographic displays, developing compelling AR experiences, and creating new human-computer interaction (HCI) paradigms—and doing it all in one cohesive product—is one of the most difficult challenges our industry has ever faced. It was so challenging that we thought we had less than a 10 percent chance of pulling it off successfully.

Until now.

A Groundbreaking Display in an Unparalleled Form Factor

At approximately 70 degrees, Orion has the largest field of view in the smallest AR glasses form factor to date. That field of view unlocks truly immersive use cases for Orion, from multitasking windows and big-screen entertainment to lifesize holograms of people—all digital content that can seamlessly blend with your view of the physical world.

For Orion, field of view was our holy grail. We were bumping up against the laws of physics and had to bend beams of light in ways they don’t naturally bend—and we had to do it in a power envelope measured in milliwatts.

Instead of glass, we made the lenses from silicon carbide—a novel application for AR glasses. Silicon carbide is incredibly lightweight, it doesn’t result in optical artifacts or stray light, and it has a high refractive index—all optical properties that are key to a large field of view. The waveguides themselves have really intricate and complex nano-scale 3D structures to diffract or spread light in the ways necessary to achieve this field of view. And the projectors are uLEDs—a new type of display technology that’s super small and extremely power efficient.

Orion is unmistakably a pair of glasses in both look and feel—complete with transparent lenses. Unlike MR headsets or other AR glasses today, you can still see each other’s real eyes and expressions, so you can be present and share the experience with the people around you. Dozens of innovations were required to get the industrial design down to a contemporary glasses form factor that you’d be comfortable wearing every day. Orion is a feat of miniaturization—the components are packed down to a fraction of a millimeter. And we managed to embed seven tiny cameras and sensors in the frame rims.

We had to maintain optical precision at one-tenth the thickness of a strand of human hair. And the system can detect tiny amounts of movement—like the frames expanding or contracting in various room temperatures—and then digitally correct for the necessary optical alignment, all within milliseconds. We made the frames out of magnesium—the same material used in F1 race cars and spacecraft—because it’s lightweight yet rigid, so it keeps the optical elements in alignment and efficiently conducts away heat.

Heating and Cooling

Once we cracked the display (no pun intended) and overcame the physics problems, we had to navigate the challenge of really powerful compute alongside really low power consumption and the need for heat dissipation. Unlike today’s MR headsets, you can’t stuff a fan into a pair of glasses. So we had to get creative. A lot of the materials used to cool Orion are similar to those used by NASA to cool satellites in outer space.

We built highly specialized custom silicon that’s extremely power efficient and optimized for our AI, machine perception, and graphics algorithms. We built multiple custom chips and dozens of highly custom silicon IP blocks inside those chips. That lets us take the algorithms necessary for hand and eye tracking as well as simultaneous localization and mapping (SLAM) technology that normally takes hundreds of milliwatts of power—and thus generates a corresponding amount of heat—and shrink it down to just a few dozen milliwatts.

And yes, custom silicon continues to play a critical role in product development at Reality Labs, despite what you may have read elsewhere. 😎

Effortless EMG

Every new computing platform brings with it a paradigm shift in the way we interact with our devices. The invention of the mouse helped pave the way for the graphical user interfaces (GUIs) that dominate our world today, and smartphones didn’t start to really gain traction until the advent of the touch screen. And the same rule holds true for wearables.

We’ve spoken about our work with electromyography, or EMG, for years, driven by our belief that input for AR glasses needs to be fast, convenient, reliable, subtle, and socially acceptable. And now, that work is ready for prime time.

Orion’s input and interaction system seamlessly combines voice, eye gaze, and hand tracking with an EMG wristband that lets you swipe, click, and scroll with ease.

It works—and feels—like magic. Imagine taking a photo on your morning jog with a simple tap of your fingertips or navigating menus with barely perceptible movements of your hands. Our wristband combines a high-performance textile with embedded EMG sensors to detect the electrical signals generated by even the tiniest muscle movements. An on-device ML processor then interprets those EMG signals to produce input events that are transmitted wirelessly to the glasses. The system adapts to you, so it gets better and more capable at recognizing the most subtle gestures over time. And today, we’re sharing more about our support for external research focused on expanding the equity and accessibility potential of EMG wristbands.

Meet the Wireless Compute Puck

True AR glasses must be wireless, and they must also be small.So we built a wireless compute puck for Orion. It takes some of the load off of the glasses, enabling longer battery life and a better form factor complete with low latency.

The glasses run all of the hand tracking, eye tracking, SLAM, and specialized AR world locking graphics algorithms while the app logic runs on the puck to keep the glasses as lightweight and compact as possible.

The puck has dual processors, including one custom designed right here at Meta, and it provides the compute power necessary for low-latency graphics rendering, AI, and some additional machine perception.

And since it’s small and sleek, you can comfortably toss the puck in a bag or your pocket and go about your business—no strings attached.

AR Experiences

Of course, as with any piece of hardware, it’s only as good as the things you can do with it. And while it’s still early days, the experiences afforded by Orion are an exciting glimpse of what’s to come.

https://youtube.com/watch?v=HkdSv3QPhNw%3F%26controls%3D0%3Fautoplay%3D0%26color%3Dwhite%26cc_lang_pref%3Den%26cc_load_policy%3D0%26enablejsapi%3D1%26frameborder%3D0%26hl%3Den%26rel%3D0%26origin%3Dhttps%253A%252F%252Fwww.meta.com

We’ve got our smart assistant, Meta AI, running on Orion. It understands what you’re looking at in the physical world and can help you with useful visualizations. Orion uses the same Llama model that powers AI experiences running on Ray-Ban Meta smart glasses today, plus custom research models to demonstrate potential use cases for future wearables development.

You can take a hands-free video call on the go to catch up with friends and family in real time, and you can stay connected on WhatsApp and Messenger to view and send messages. No need to pull out your phone, unlock it, find the right app, and let your friend know you’re running late for dinner—you can do it all through your glasses.

https://youtube.com/watch?v=el7lUVvu8Bo%3F%26controls%3D0%3Fautoplay%3D0%26color%3Dwhite%26cc_lang_pref%3Den%26cc_load_policy%3D0%26enablejsapi%3D1%26frameborder%3D0%26hl%3Den%26rel%3D0%26origin%3Dhttps%253A%252F%252Fwww.meta.com

You can play shared AR games with family on the other side of the country or with your friend on the other side of the couch. And Orion’s large display lets you multitask with multiple windows to get stuff done without having to lug around your laptop.

The experiences available on Orion today will help chart the roadmap for our consumer AR glasses line in the future. Our teams will continue to iterate and build new immersive social experiences, alongside our developer partners, and we can’t wait to share what comes next.

A Purposeful Product Prototype

While Orion won’t make its way into the hands of consumers, make no mistake: This is not a research prototype. It’s the most polished product prototype we’ve ever developed—and it’s truly representative of something that could ship to consumers. Rather than rushing to put it on shelves, we decided to focus on internal development first, which means we can keep building quickly and continue to push the boundaries of the technology and experiences.

And that means we’ll arrive at an even better consumer product faster.

What Comes Next

Two major obstacles have long stood in the way of mainstream consumer-grade AR glasses: the technological breakthroughs necessary to deliver a large display in a compact glasses form factor and the necessity of useful and compelling AR experiences that can run on those glasses. Orion is a huge milestone, delivering true AR experiences running on reasonably stylish hardware for the very first time.

Now that we’ve shared Orion with the world, we’re focused on a few things:

• Tuning the AR display quality to make the visuals even sharper
• Optimizing wherever we can to make the form factor even smaller
• Building at scale to make them more affordable

In the next few years, you can expect to see new devices from us that build on our R&D efforts. And a number of Orion’s innovations have been extended to our current consumer products today as well as our future product roadmap. We’ve optimized some of our spatial perception algorithms, which are running on both Meta Quest 3S and Orion. While the eye gaze and subtle gestural input system was originally designed for Orion, we plan to use it in future products. And we’re exploring the use of EMG wristbands across future consumer products as well.

Orion isn’t just a window into the future—it’s a look at the very real possibilities within reach today. We built it in our pursuit of what we do best: helping people connect. From Ray-Ban Meta glasses to Orion, we’ve seen the good that can come from letting people stay more present and empowered in the physical world, even while tapping into all the added richness the digital world has to offer.

We believe you shouldn’t have to choose between the two. And with the next computing platform, you won’t have to.

---

For more information on Orion, check out these blog posts:

• Orion’s Compute Puck: The Story Behind the Device that Helped Make Our AR Glasses Possible
• Zero to One: How Our Custom Silicon & Chips Are Revolutionizing AR
• Crystal Clear: Our Silicon Carbide Waveguides & the Path to Orion’s Large FoV

Last year at Connect, we unveiled Orion—our first true pair of AR glasses. The culmination of our work at Reality Labs over the last decade, Orion combines the benefits of a large holographic display and personalized AI assistance in a comfortable, all-day wearable form factor. It got some attention for its industry-leading field of view, silicon carbide waveguides, uLED projectors, and more. But today, we’re turning our attention to an unsung hero of Orion: the compute puck.

Designed to easily slip into your pocket or bag so you can bring it just about anywhere as you go about your day, the puck offloads Orion’s processing power to run application logic and enable a more compelling, smaller form factor for the glasses. It connects wirelessly to the glasses and EMG wristband for a seamless experience.

Set it and forget it, right?

But the puck’s backstory—even as a prototype—is involved, with a dramatic arc you’d never guess at from its appearance.

“When you’re building something like this, you start getting into the limits of physics,” explains Director of Product Management Rahul Prasad. “For the last 50 years, Moore’s Law has made everything smaller, faster, and lower power. The problem is that now you’re starting to hit limits on how much heat you can dissipate, how much battery you can compress, and how much antenna performance you can fit into a particular sized object.”

While hindsight may be 20/20, the puck’s potential wasn’t immediately obvious. When you’re the first to build something, you need to explore every possibility—leaving no stone unturned. How do you build something that some might think of as an undesirable accessory rather than a critical part of the package?

“We knew the puck was an extra device we were asking people to carry,” notes Product Design Engineering Manager Jared Van Cleave, “so we explored how to turn the bug into a feature.”

Ultimately, that ethos paid off in spades as the puck squeezes a lot of compute (and even more Meta-designed custom silicon for AI and machine perception) into a small size. This was instrumental in helping Orion go from the realm of science fiction into reality.

“If you didn’t have the puck, you wouldn’t be able to have the experiences that Orion offers in its form factor—period,” says Prasad. “A good AR experience demands really high performance: high frame rates, extremely low latency, fine-grained wireless and power management, etc. The puck and the glasses need to be co-designed to work really closely together, not just at the app layer, but also at the operating system, firmware, and hardware layers. And even if one were to co-design a smartphone to work with AR glasses, the demanding performance requirements would drain the phone battery and suck away compute capacity from phone use cases. On the other hand, the puck has its own high-capacity battery, high-performance SoC, and a custom Meta-designed AI co-processor optimized for Orion.”

Of course, the puck wasn’t designed overnight. It required years of iterative work.

“We didn’t know how people would want to interact with Orion from an input perspective—there was nothing we could draft off of in-market,” says Product Manager Matt Resman. “If you look at our early glasses prototypes, they were these massive headsets that weighed three or four pounds. And when you’re trying to build a product, it’s really difficult to understand the user experience if you’re not in the right form factor. With the puck, we were able to prototype very quickly to start understanding how people would use it.”

Codenamed Omega in the early days, the puck was initially envisioned by what was then Oculus Research as an Ω-shaped band that would go around the user’s neck and be hard-wired to the glasses…

… until some new innovations by the Reality Labs wireless team, among other things, allowed them to cut the cord. That enabled a more handheld or pocketable / in-bag form factor, which opened up a lot of possibilities.

“At that point, augmented reality calling was still a primary use case,” Van Cleave explains. “The puck was where the holographic videos would be anchored. You’d put it down on the table with the sensor bench facing you, imaging you, and then projecting who you were speaking to from the puck’s surface for the call.”

“Orion is about connecting people and bringing us together,” says Resman. “One of the initial concepts for the puck was to help create this sense of presence with other people and enable this richer form of communication.”

“It’s unlike anything you’ve ever seen—the device has the potential to create really fun and unique interactions,” adds Industrial Designer Emron Henry. “The user experience feels a bit like unleashing a genie from a bottle, where holograms seamlessly emerge from and dissolve back into the device.”

As you can probably tell by now, there’s more potential inside the puck than what ultimately got turned on as a feature. In addition to the sensors and cameras that would’ve been used for AR calling, the puck has haptics and 6DOF sensors that could enable it to be used as a tracked controller to select and manipulate virtual objects and play AR games. The team also explored capacitive and force touch input so the puck could serve as a gamepad when held in both portrait and landscape mode.

“We would talk about the need to carry this extra thing,” says Van Cleave. “How do we make it more useful? There was a whole workstream around it. And at one point, the hypothesis was that AR gaming was going to be this killer use case.”

Early on, we knew we needed to explore the possibilities of the puck doing things that phones cannot. Eventually, we landed on using eye gaze, EMG, and hand tracking for AR games, like Stargazer and Pong, but we prototyped various demos and games that used the puck as a controller in the early days of Orion.

0:00 / 0:00

0:00 / 0:00

0:00 / 0:00

Rendered explorations of the puck as a 6DOF controller for AR games like Pong.

“Because it’s not a phone, that gave us a lot of design freedom,” adds Prasad. “It can be thicker, I can make it more rounded so it fits comfortably in your hand as a controller. It’s pretty incredible that the puck is actually smaller than the average phone, but it’s more powerful than a phone because it has a Meta-designed co-processor for AI and machine perception.”

The team dug into the question of how a quality AR game might differ from a MR or console game. That meant exploring different affordances for an AR game controller, including joysticks, physical button layouts, trigger buttons, and more.

“We didn’t end up actually building any of that stuff,” Van Cleave notes. “We prototyped it, but we didn’t ever go for a full build. We wanted to keep it simple. We wanted to do these soft interfaces but not make physical, mechanical buttons.”

And while the sensors and haptics weren’t turned on in the finished product prototype, they served an integral function during development, letting teams file bugs by simply tapping on the top of the device a few times to trigger a bug report.

As AI began to take center stage as a key use case, compute came into increasingly sharp focus. In the end, the puck is home to Orion’s wireless connectivity, computing power, and battery capacity—a huge technical feat in itself—all of which helps reduce the weight and form factor of the glasses while dissipating a lot more thermals by virtue of its surface area.

Throughout its evolution, one thing has remained the same: There’s more to the puck than meets the untrained eye. Embracing unconventional ideas allowed our teams to explore, push boundaries, and build the future.

“We’re defining a category that doesn’t quite exist yet,” notes Henry. “As you’d expect with R&D, there were starts and stops along the way. How will users expect to interact with holograms? Would they prefer to use an AR remote or is hand tracking, eye gaze, and EMG sufficient for input? What feels intuitive, low-friction, familiar, and useful?”

“Rather than looking at the puck as just a rock, we asked ourselves what else we could provide to further differentiate it from phones and justify why you’d want to carry it around,” acknowledges Resman. “Does its raw compute power and practical design—which helped unlock a glasses form factor that you can realistically wear around all day—ultimately offer enough value? It’s our job to help answer these questions.”

“Early on, we shifted gears to focus on what we can do that a phone can’t do,” Van Cleave adds. “Phones need to have the screen, they need to have the certain physical button layout that users expect. And we don’t have those constraints. Our compute puck can be whatever we want it to be.”

---

For more information on Orion, check out these blog posts:

• Crystal Clear: Our Silicon Carbide Waveguides & the Path to Orion’s Large FoV
• Zero to One: How Our Custom Silicon & Chips Are Revolutionizing AR

TL;DR:

• Today at Connect, Mark Zuckerberg unveiled Orion—our first pair of true AR glasses, previously codenamed Project Nazare.
• With an industry-leading field of view, silicon carbide lenses, intricate waveguides, uLED projectors, and more, Orion is our most advanced and most polished product prototype to date.
• Now, we’re continuing product optimizations and lowering costs as we drive toward a scalable consumer device that will revolutionize the way people interact with the world.

---

“We are building AR glasses.”

Five simple words, spoken five years ago. With them, we planted a flag on our vision of a future where we no longer have to make the false choice between a world of information at our fingertips and the physical world all around us.

And now, five years later, another five words that stand to once again change the game:

We have built AR glasses.

https://youtube.com/watch?v=2CJsnyS8u3c%3Fsi%3DTr77gOKEq3PnA7-k%26controls%3D0%3Fautoplay%3D0%26color%3Dwhite%26cc_lang_pref%3Den%26cc_load_policy%3D0%26enablejsapi%3D1%26frameborder%3D0%26hl%3Den%26rel%3D0%26origin%3Dhttps%253A%252F%252Fwww.meta.com

At Meta, our mission is simple: give people the power to build community and bring the world closer together. And at Reality Labs, we build tools that help people feel connected anytime, anywhere. That’s why we’re working to build the next computing platform that puts people at the center so they can be more present, connected, and empowered in the world.

Ray-Ban Meta glasses have demonstrated the power of giving people hands-free access to key parts of their digital lives from their physical ones. We can talk to a smart AI assistant, connect with friends, and capture the moments that matter—all without ever having to pull out a phone. These stylish glasses fit seamlessly into our everyday lives, and people absolutely love them.

Yet while Ray-Ban Meta opened up an entirely new category of display-less glasses super-charged by AI, the XR industry has long dreamt of true AR glasses—a product that combines the benefits of a large holographic display and personalized AI assistance in a comfortable, stylish, all-day wearable form factor.

And today, we’ve pushed that dream closer to reality with the unveiling of Orion, which we believe is the most advanced pair of AR glasses ever made. In fact, it might be the most challenging consumer electronics device produced since the smartphone. Orion is the result of breakthrough inventions in virtually every field of modern computing—building on the work we’ve been doing at Reality Labs for the last decade. It’s packed with entirely new technologies, including the most advanced AR display ever assembled and custom silicon that enables powerful AR experiences to run on a pair of glasses using a fraction of the power and weight of an MR headset.

Orion’s input system seamlessly combines voice, eye gaze, and hand tracking with an EMG wristband that lets you swipe, click, and scroll while keeping your arm resting comfortably by your side, letting you stay present in the world and with the people around you as you interact with rich digital content.

Beginning today at Connect and continuing throughout the year, we’re opening up access to our Orion product prototype for Meta employees and select, external audiences so our development team can learn, iterate, and build towards our consumer AR glasses product line, which we plan to begin shipping in the near future.

Why AR Glasses?

There are three primary reasons why AR glasses are key to unlocking the next great leap in human-oriented computing.

1. They enable digital experiences that are unconstrained by the limits of a smartphone screen. With large holographic displays, you can use the physical world as your canvas, placing 2D and 3D content and experiences anywhere you want.
2. They seamlessly integrate contextual AI that can sense and understand the world around you in order to anticipate and proactively address your needs.
3. They’re lightweight and great for both indoor and outdoor use—and they let people see each other’s real face, real eyes, and real expressions.

That’s the north star our industry has been building towards: a product combining the convenience and immediacy of wearables with a large display, high-bandwidth input, and contextualized AI in a form factor that people feel comfortable wearing in their daily lives.

Compact Form Factor, Compounded Challenges

For years, we’ve faced a false choice—either virtual and mixed reality headsets that enable deep, immersive experiences in a bulky form factor or glasses that are ideal for all-day use but don’t offer rich visual apps and experiences given the lack of a large display and corresponding compute power.

But we want it all, without compromises. For years, we’ve been hard at work to take the incredible spatial experiences afforded by VR and MR headsets and miniaturizing the technology necessary to deliver those experiences into a pair of lightweight, stylish glasses. Nailing the form factor, delivering holographic displays, developing compelling AR experiences, and creating new human-computer interaction (HCI) paradigms—and doing it all in one cohesive product—is one of the most difficult challenges our industry has ever faced. It was so challenging that we thought we had less than a 10 percent chance of pulling it off successfully.

Until now.

A Groundbreaking Display in an Unparalleled Form Factor

At approximately 70 degrees, Orion has the largest field of view in the smallest AR glasses form factor to date. That field of view unlocks truly immersive use cases for Orion, from multitasking windows and big-screen entertainment to lifesize holograms of people—all digital content that can seamlessly blend with your view of the physical world.

For Orion, field of view was our holy grail. We were bumping up against the laws of physics and had to bend beams of light in ways they don’t naturally bend—and we had to do it in a power envelope measured in milliwatts.

Instead of glass, we made the lenses from silicon carbide—a novel application for AR glasses. Silicon carbide is incredibly lightweight, it doesn’t result in optical artifacts or stray light, and it has a high refractive index—all optical properties that are key to a large field of view. The waveguides themselves have really intricate and complex nano-scale 3D structures to diffract or spread light in the ways necessary to achieve this field of view. And the projectors are uLEDs—a new type of display technology that’s super small and extremely power efficient.

Orion is unmistakably a pair of glasses in both look and feel—complete with transparent lenses. Unlike MR headsets or other AR glasses today, you can still see each other’s real eyes and expressions, so you can be present and share the experience with the people around you. Dozens of innovations were required to get the industrial design down to a contemporary glasses form factor that you’d be comfortable wearing every day. Orion is a feat of miniaturization—the components are packed down to a fraction of a millimeter. And we managed to embed seven tiny cameras and sensors in the frame rims.

We had to maintain optical precision at one-tenth the thickness of a strand of human hair. And the system can detect tiny amounts of movement—like the frames expanding or contracting in various room temperatures—and then digitally correct for the necessary optical alignment, all within milliseconds. We made the frames out of magnesium—the same material used in F1 race cars and spacecraft—because it’s lightweight yet rigid, so it keeps the optical elements in alignment and efficiently conducts away heat.

Heating and Cooling

Once we cracked the display (no pun intended) and overcame the physics problems, we had to navigate the challenge of really powerful compute alongside really low power consumption and the need for heat dissipation. Unlike today’s MR headsets, you can’t stuff a fan into a pair of glasses. So we had to get creative. A lot of the materials used to cool Orion are similar to those used by NASA to cool satellites in outer space.

We built highly specialized custom silicon that’s extremely power efficient and optimized for our AI, machine perception, and graphics algorithms. We built multiple custom chips and dozens of highly custom silicon IP blocks inside those chips. That lets us take the algorithms necessary for hand and eye tracking as well as simultaneous localization and mapping (SLAM) technology that normally takes hundreds of milliwatts of power—and thus generates a corresponding amount of heat—and shrink it down to just a few dozen milliwatts.

And yes, custom silicon continues to play a critical role in product development at Reality Labs, despite what you may have read elsewhere. 😎

Effortless EMG

Every new computing platform brings with it a paradigm shift in the way we interact with our devices. The invention of the mouse helped pave the way for the graphical user interfaces (GUIs) that dominate our world today, and smartphones didn’t start to really gain traction until the advent of the touch screen. And the same rule holds true for wearables.

We’ve spoken about our work with electromyography, or EMG, for years, driven by our belief that input for AR glasses needs to be fast, convenient, reliable, subtle, and socially acceptable. And now, that work is ready for prime time.

Orion’s input and interaction system seamlessly combines voice, eye gaze, and hand tracking with an EMG wristband that lets you swipe, click, and scroll with ease.

It works—and feels—like magic. Imagine taking a photo on your morning jog with a simple tap of your fingertips or navigating menus with barely perceptible movements of your hands. Our wristband combines a high-performance textile with embedded EMG sensors to detect the electrical signals generated by even the tiniest muscle movements. An on-device ML processor then interprets those EMG signals to produce input events that are transmitted wirelessly to the glasses. The system adapts to you, so it gets better and more capable at recognizing the most subtle gestures over time. And today, we’re sharing more about our support for external research focused on expanding the equity and accessibility potential of EMG wristbands.

Meet the Wireless Compute Puck

True AR glasses must be wireless, and they must also be small.So we built a wireless compute puck for Orion. It takes some of the load off of the glasses, enabling longer battery life and a better form factor complete with low latency.

The glasses run all of the hand tracking, eye tracking, SLAM, and specialized AR world locking graphics algorithms while the app logic runs on the puck to keep the glasses as lightweight and compact as possible.

The puck has dual processors, including one custom designed right here at Meta, and it provides the compute power necessary for low-latency graphics rendering, AI, and some additional machine perception.

And since it’s small and sleek, you can comfortably toss the puck in a bag or your pocket and go about your business—no strings attached.

AR Experiences

Of course, as with any piece of hardware, it’s only as good as the things you can do with it. And while it’s still early days, the experiences afforded by Orion are an exciting glimpse of what’s to come.

https://youtube.com/watch?v=HkdSv3QPhNw%3F%26controls%3D0%3Fautoplay%3D0%26color%3Dwhite%26cc_lang_pref%3Den%26cc_load_policy%3D0%26enablejsapi%3D1%26frameborder%3D0%26hl%3Den%26rel%3D0%26origin%3Dhttps%253A%252F%252Fwww.meta.com

We’ve got our smart assistant, Meta AI, running on Orion. It understands what you’re looking at in the physical world and can help you with useful visualizations. Orion uses the same Llama model that powers AI experiences running on Ray-Ban Meta smart glasses today, plus custom research models to demonstrate potential use cases for future wearables development.

You can take a hands-free video call on the go to catch up with friends and family in real time, and you can stay connected on WhatsApp and Messenger to view and send messages. No need to pull out your phone, unlock it, find the right app, and let your friend know you’re running late for dinner—you can do it all through your glasses.

https://youtube.com/watch?v=el7lUVvu8Bo%3F%26controls%3D0%3Fautoplay%3D0%26color%3Dwhite%26cc_lang_pref%3Den%26cc_load_policy%3D0%26enablejsapi%3D1%26frameborder%3D0%26hl%3Den%26rel%3D0%26origin%3Dhttps%253A%252F%252Fwww.meta.com

You can play shared AR games with family on the other side of the country or with your friend on the other side of the couch. And Orion’s large display lets you multitask with multiple windows to get stuff done without having to lug around your laptop.

The experiences available on Orion today will help chart the roadmap for our consumer AR glasses line in the future. Our teams will continue to iterate and build new immersive social experiences, alongside our developer partners, and we can’t wait to share what comes next.

A Purposeful Product Prototype

While Orion won’t make its way into the hands of consumers, make no mistake: This is not a research prototype. It’s the most polished product prototype we’ve ever developed—and it’s truly representative of something that could ship to consumers. Rather than rushing to put it on shelves, we decided to focus on internal development first, which means we can keep building quickly and continue to push the boundaries of the technology and experiences.

And that means we’ll arrive at an even better consumer product faster.

What Comes Next

Two major obstacles have long stood in the way of mainstream consumer-grade AR glasses: the technological breakthroughs necessary to deliver a large display in a compact glasses form factor and the necessity of useful and compelling AR experiences that can run on those glasses. Orion is a huge milestone, delivering true AR experiences running on reasonably stylish hardware for the very first time.

Now that we’ve shared Orion with the world, we’re focused on a few things:

• Tuning the AR display quality to make the visuals even sharper
• Optimizing wherever we can to make the form factor even smaller
• Building at scale to make them more affordable

In the next few years, you can expect to see new devices from us that build on our R&D efforts. And a number of Orion’s innovations have been extended to our current consumer products today as well as our future product roadmap. We’ve optimized some of our spatial perception algorithms, which are running on both Meta Quest 3S and Orion. While the eye gaze and subtle gestural input system was originally designed for Orion, we plan to use it in future products. And we’re exploring the use of EMG wristbands across future consumer products as well.

Orion isn’t just a window into the future—it’s a look at the very real possibilities within reach today. We built it in our pursuit of what we do best: helping people connect. From Ray-Ban Meta glasses to Orion, we’ve seen the good that can come from letting people stay more present and empowered in the physical world, even while tapping into all the added richness the digital world has to offer.

We believe you shouldn’t have to choose between the two. And with the next computing platform, you won’t have to.

---

For more information on Orion, check out these blog posts:

• Orion’s Compute Puck: The Story Behind the Device that Helped Make Our AR Glasses Possible
• Zero to One: How Our Custom Silicon & Chips Are Revolutionizing AR
• Crystal Clear: Our Silicon Carbide Waveguides & the Path to Orion’s Large FoV