If the awesome ARKit-powered measuring tape hasn't knocked your socks off, how about PCalc, the best scientific calculator available on App Store, in augmented reality?
There's certainly no shortage of impressive demos of awesome augmented reality apps made possible by Apple's new ARKit framework in iOS 11, but this new proof-of-concept app from independent game developer Duncan Walker certainly takes things up a notch.
An extension of Metaio's patent that was assigned to Apple following its acquisition of the company in 2015, Apple's new patent application published Thursday by the United States Patents and Trademark Office (USPTO) details potential applications for augmented reality while illustrating a rumored head-mounted display Apple is said to be working on.
Windows giant Microsoft will outfit the next version of its mixed reality HoloLens goggles with an extra AI processor that will let the device recognize speech and images in real time, without having to send the data back to the cloud for analysis.
iOS 11 introduced ARKit, a new framework for developers designed to shorten development times in terms of building augmented reality experiences for iPhone and iPad.
Because ARKit does most of the heavy lifting in terms of plane detection, super accurate tracking, lighting estimation and more, apps powered by it are unsupported on older devices.
Why wait until 2018 or beyond for a rumored Apple headset when you can immerse yourself in augmented reality right now, and without breaking the bank? Meet Mira Prism, a new iPhone-connected headset that provides augmented reality experiences for just $99.
The product is standalone and requires no plugs, computers or wires.
You just side your iPhone into the headset to begin exploring the wonders of interactive holographic content. It should be mentioned that Mira Prism does not put your iPhone's screen right in front of your eyes, like Google's Cardboard does.
Instead, your iPhone faces the inside of the headset and the front glass lenses facing you basically reflect computer imagery projected onto their surface by a pair of mirrors. The accessory has a 60-degree field of view and supports a resolution of 1,334-by-750 pixels.
https://www.youtube.com/watch?v=w76ESU3XHOg
While the headset only supports a small number of launch titles created specifically for it, they'll be releasing an SDK to let anyone create compatible augmented reality apps and games.
An accompanying app features spectator mode so those without a headset can see what others are seeing in augmented reality through the headset on their own iPhone or iPad.
Engadget took the gadget for a spin and came away impressed:
I played a holographic game that involved maneuvering a character through a maze, which relied on the controller’s motion controls.
Another game had me spinning around in my chair to destroy asteroids hovering all around me. I was particularly surprised how well Prism tracked virtual objects in augmented reality, even though it doesn’t have any spatial mapping technology like HoloLens and Meta.
Mira Prism ships with a remote control for motion-based games, a carrying case and a lens cover, and is compatible with any iPhone from iPhone 6s onward. The biggest drawback of Prism is the fact it won't work with any ARKit-optimized app, just stuff made specifically for it.
If you're interested in this product, pre-order it today for $99 from mirareality.com because it will retail for $150 once officially released in time for 2017 holiday.
Unlike virtual reality which completely immerses you in computer-generated images, augmented reality superimposes virtual objects on top of your world rather than close it out.
Mixed reality, on the other hand, (sometimes called hybrid reality) merges the two worlds so that physical and digital objects co-exist and interact in real time.
A new recently published demo combines an ARKit-driven app running on the Unity 3D engine on an iPad Pro with real-time input from HTC's Vive virtual reality headset and controller. Created by NY-based virtual reality developer Normal VR, the app features a cute avatar drawing a virtual painting in the middle of the company's office.
What's really interesting about it is the fact that a person wearing a HTC Vive is revealed when the camera pans right, proving that the app captures the moves of the person in real-time and projects the resulting actions in their real world through augmented reality.
“This is going from East Coast to West Coast (server) and back. We definitely do some extrapolation to account for ping,” developers wrote. You could easily imagine the possibilities for remote interactions between people across the globe in mixed reality.
Blobbing in the studio today w/ the Vive + ARKit. Definitely some huge mixed reality potential here. #arkit #vr #indiedev #gamedev pic.twitter.com/C1zANBuSrx
— Normal (@normalvr) July 10, 2017
Not sure about you, but I'm really impressed by this demo.
There's a lot going on here as this app brings objects from the virtual world into the real one, in real-time—not only does the avatar realistically replicates the person's actions, it shows the digital painting on top of the real world as it's being created, updating it constantly.
Welcome to the future of mixed reality!
We showed you many demos powered by Apple's ARKit framework for building augmented reality apps on iPhone and iPad, ranging from a simple but awesome measuring tape to home decor shopping, accurate room measurement, Tic Tac Toe, food ordering and more.
Even Ikea has jumped on the AR bandwagon by partnering with Apple on an ARKit-powered app that will let you try out virtual furniture in your home, with support for in-app ordering.
The beauty of ARKit is that it does all the heavy lifting, allowing developers to focus on app design rather than deal with things like plane detection, lighting estimation, tracking, etc.
Using computer vision and other techniques, ARKit does all that by analyzing live camera feed and data from iPhone or iPad's built-in sensors. ARKit requires the A9 or A10 chip, meaning augmented reality apps will require an iPhone 6s or newer or one of the latest iPad models.
Apple's new ARKit framework for building augmented reality experiences for compatible iPhone and iPad devices is off to a great start and already Facebook is taking notice, according to a new report Thursday from Bloomberg's Mark Gurman.
The author claims that Facebook has been hard at work developing an inexpensive headset, code-named “Pacific”, that is expected to bring virtual reality experiences to the masses without requiring a beefy computer or a compatible smartphone.
“The idea is that someone will be able to pull the headset out of their bag and watch movies on a flight just the way you can now with a phone or tablet,” reads the article.
It should be priced aggressively at $200 and release at some point next year, representing “an entirely new category”. According to people familiar with the plans, the device will provide a similar interface to Samsung’s VR Gear that users could control by a wireless remote.
The headset should be powered by Qualcomm's Snapdragon mobile chip that would make it superior to Samsung's Gear VR headset in terms of gaming in virtual reality. Unlike the current Oculus Rift hardware, the upcoming gizmo won't include positional tracking technology.
An excerpt from the article:
This means that the device won’t be able to tell where its user is spatially, which is useful for tasks like virtual rock climbing. A future version of the product will have that technology, according to a person familiar with the plans.
According to sources, the headset will let users play immersive games, watch video, use social networking apps and so forth. It resembles a more compact version of the current Oculus Rift and will be lighter than Samsung’s Gear VR headset.
Handset maker Xiaomi and its manufacturers are said to build 2018's Oculus-branded device.
And later this year, Facebook allegedly plans to announce a more affordable wireless headset that it is betting will popularize virtual reality “the way Apple did the smartphone”.
Oculus spokesman Alan Cooper said via email:
We don’t have a product to unveil at this time, however we can confirm we’re making several significant technology investments in the standalone VR category.
Facebook's said it’s also working on yet another device, code-named “Santa Cruz” and best described as a wireless Oculus Rift “with the full power of the original device sans PC.“
Facebook acquired Kickstarter-funded Oculus startup in 2014 for about $2 billion.
IDC estimated that Samsung leads the pack in terms of VR device shipments with 22 percent of the global market for VR devices, followed by Sony, HTC and Facebook's Oculus Rift with about five percent of the market, or less than 100,000 units sold.
A great little ARKit-powered hackathon project by Melbourne, Australia developer Jacqui Hyslop of RedBubble shows off the tremendous potential of Apple's new framework for building augmented reality experiences for iPhone and iPad with iOS 11.
The simple proof-of-concept app allows the user to place virtual cushions (and other home decor items) on a couch and other pieces of furniture in their real world, but without the psychedelic surface-shifting effect plaguing many other AR platforms.
This app clearly proves that home decor shopping in augmented reality beats having to drive to a store just to see how that cushion would look like on a bed that isn't yours in a showroom that shares little resemblance to your own home.
The video demonstration, embedded below for your viewing pleasure, highlights the full sophistication of ARKit's reliable plane and object detection and precision tracking. It's remarkable that all of this is happening so fluidly without any special hardware involved.
https://www.youtube.com/watch?v=bwNAykf4IKw
ARKit uses machine learning, artificial intelligence, computer vision and sophisticated algorithms to analyze live camera feed and “see” the world around you. It also taps into data from on-board device sensors, such as the accelerometer and gyroscope, to detect horizontal planes in your real world, determine local lighting conditions and so forth.
Those kinds of ARKit-powered apps should be even better on iPhone 8 due to the rumored rear-facing 3D laser sensor that will allow the device to accurately sense depth and map objects.
Be sure to watch another sophisticated ARKit demo showing off an app that basically allows the user to map out the floorspace by walking around a room. It calculates total footage of the room and may even produce detailed blueprints that could be exported to CAD apps.
And here it is.
https://www.youtube.com/watch?v=iw9MPZoPqCQ
The future of furniture shopping looks bright, too, as furniture giant Ikea has partnered with Apple on an ARKit-powered app of its own that promises to let users try out virtual furniture at home before placing an actual order right inside the app.
Just imagine the possibilities: with Ikea's app, you could make sure that the bookshelf you've been eyeing for some time would fit in that corner—again, without being bothered to actually, you know, visit one of their stores.
Are you excited for ARKit-powered apps?
If the augmented reality (AR) tape measure app we recently featured has gotten you excited for the possibilities that Apple's ARKit framework brings to developers, you're wholeheartedly recommended to watch this video demonstration of an upcoming ARKit-driven app that will let you take precise room measurement by pointing your iPhone's camera at the corners.
The app will help generate 3D models and automatically calculate the footage for the user. It's a really cool-looking demo and I can't wait to try this app out when it releases.
Watch the demonstration video below.
https://www.youtube.com/watch?v=iw9MPZoPqCQ
The app was created by a company called Smart Pictures, which provides the Measurement Cloud system that enables dimensioning intelligence for the interior world.
Here's a recent ARKit-powered tape measure demo that set the Internet on fire.
https://www.youtube.com/watch?v=z7DYC_zbZCM
This level of precision is made possible by ARKit's accurate plane detection and reliable object tracking. On existing devices, the ARKit framework leverages data from on-board sensors and uses computer vision algorithms constantly analyzing live camera feed.
ARKit-driven apps should be even better on iPhone 8 because Apple's OLED iPhone is said to feature a dedicated 3D laser sensor on both its front and back for even better augmented reality features and faster, more precise autofocus.
Fast Company is out with a new report that basically reiterates what Barclays and KGI Securities said before: that Apple's OLED iPhone will actually feature two 3D sensors, one on the front and the other out the back. In addition, the device could feature completely waterproof buttons on its side that respond with haptic feedback.
According to the publication's sources, the rear 3D sensor will help iPhone 8 provide improved camera autofocus and enhanced augmented reality features, likely stemming from even more accurate object tracking and better depth detection in ARKit-powered apps.
Like the front-facing 3D sensor, the rear one is said to use a vertical-cavity surface-emitting (VSCEL) laser system that also includes a lens, a detector sensor and a dedicated chip. The VCSEL system would map objects in 3D by calculating the distance the light travels from the laser to the target and back to the sensor, using so-called time-of-flight measurement.
Suppliers Lumentum, Finisar and II-VI will reportedly be providing the VCSEL lasers for the device. STMicro, Infineon or AMS should provide the time-of-flight sensor. Apple could buy the whole modular system from either LG Innotek, STMicro, AMS or Foxconn, added the source.
The system costs about $2 per phone, as per the publication.
The laser-based sensor will also enable a faster and more accurate autofocus than the phase detection autofocus used on current iPhones. That's because the dedicated rear 3D sensor will determine precise object depth by measuring the time it takes the laser light beams to bounce off objects and return to the sensor.
This will let iPhone 8 focus the camera lens on the desired object in milliseconds. Laser autofocus systems are already used in phones from Google, Huawei, OnePlus and Asus.
Apple's ARKit framework is slowly but surely emerging as one of the best new features in iOS 11. Many developers have built everything from virtual tape measures and Minecraft to ballerinas made out of wood dancing on floors. It's remarkable that most of the ARKit demos we've seen so far were built in a matter of hours or days, not weeks or months.
Today, we want to highlight a few additional ARKit demos that we've curated. These videos highlight ARKit's incredibly reliable and accurate tracking features that don't require any special hardware beyond the sensors and the camera already present in your iPhone or iPad.
First up, we have this demo showing jumping between different planes.
https://www.youtube.com/watch?v=pj3PbRTgpQk
ARKit automatically detects horizontal surfaces, called planes, such as tables and floors, and can track and place objects on smaller feature points as well. This is all handled automatically, with uncanny precision, using only data from your iOS device's camera and sensors.
The following pair of videos demonstrate a virtual character interacting with the environment by autonomously jumping a flight of stairs and between different surfaces of the real world.
https://www.youtube.com/watch?v=vozrtqe7MZY
https://www.youtube.com/watch?v=o7REJj_bN-c
Interactions between virtual objects and real people are easy as a pie with ARKit.
https://www.youtube.com/watch?v=RdAqD4uIbX0
The Tracking Monster demo, seen below, uses ARKit and the Unity engine to track a monster with the dynamically updated shadows based on changing lighting conditions in the real world.
https://www.youtube.com/watch?v=_IYT-OCWqDg
Maze games will never be the same!
Speaking of games, here's Tic-Tac-Tio in augmented reality, developed By Bjarne Lundgren.
https://www.youtube.com/watch?v=IBBq473vuMo
And last but not least, Mixed Reality Design posted the following example of an augmented reality app depicting an alien vessel hovering ominously above a construction site.
Hollywood will soon invade your smartglasses airspace → fact https://t.co/jLYm1YcBW2 pic.twitter.com/to2qqfFIVr
— Mixed Reality Design (@MixedrealityD) July 4, 2017
Head over to the Mixed Reality Design's Twitter account for more AR examples like this.
While it's not entirely clear that this particular demo uses ARKit, it does highlight the possibilities for AR movie trailers that could be coming soon to your phone.
Be sure to check out other interesting ARKit apps and demos, including an upcoming furniture-ordering app from Ikea, a measuring tape that blew up on the web, an ARKit-powered VR mode in Maps, an inter-dimensional portal and much more.
“I think there is a gigantic runway that we have here with the iPhone and the iPad,” Apple executive Greg Joswiak said of ARKit in a recent interview with The Australian. “The fact we have a billion of these devices out there is quite an opportunity for developers.”
How do you like these ARKit demos? Which one is your favorite, and why? Chime in with your thoughts and observations in the comments section below.