Publications & Patents
Patents and papers, grouped by the problem they solve — from cameras hidden behind displays to drones that paint with light. Every item carries the one-line reason it matters.
Topic 01
The flagship thread: cameras, sensors, and eyes woven into the display itself — the 0-to-1 work that shipped in 2026, and the sensing still to come.
US 11,294,422 B1 · US 11,592,864 · US 11,853,117 · US 12,124,291
The disappearing camera. The 0-to-1 architecture — apertured display, behind-display sensor, computational restoration — that shipped in the iPhone Duo’s hidden FaceTime camera and the iPhone 18 Pro Max’s under-display Face ID camera. Cited as prior art by 27 later publications across the industry.
US 2018/0082482 A1 · EP 3,488,315 B1 (granted 2021)
The sensing behind passthrough AR. World-facing and user-facing sensors unified in one display system — the architecture that lets a headset see the room and the wearer at once.
US 2019/0221044 A1 · US 11,217,021 B2 (granted 2022)
Sensors living inside the screen. Integrating cameras and sensors into the display stack itself — the display stops being just an output and becomes an input.
US 11,550,408 B1 · US 12,105,897 B2
A stylus that truly sees color. Multi-channel optical sensing samples real-world surfaces — point at anything and capture its exact color, not an approximation.
Gaze tracking with no camera. The sensor disappears entirely — eye tracking that leaves only the signal.
The display becomes the sensor. Photodiodes woven into the screen image the eye from inside it — sensing without a visible sensor.
US 10,817,594 B2 · US 11,036,844 B2
Your wrist is the password. A light-field camera on a wearable that authenticates from the dorsal forearm — biometrics with no fingerprint sensor and nothing to touch.
The whole light field, in one shot. Captures ray direction as well as intensity — the raw material for refocus-after-capture and single-shot depth.
WO 2024/238704 A1 · US 2024/0385454 A1
Building the headset, piece by piece. Head-mounted display systems from the founding years of the Vision Products Group.
Topic 02
Teaching cameras to see in depth — from light-field regularization to stereo-defocus fusion, including his most-cited work.
US 9,898,856 (2018) · US 10,540,806 B2 · WO 2015/048694 A2
His most-cited work. Uses depth to undo wide-angle perspective distortion — the reason faces stop looking warped at the edge of the frame.
US 10,574,905 B2 · US 10,089,740 · WO 2015/134996 A1
Making light-field depth trustworthy. Depth regularization that cleaned up noisy light-field depth — the foundation of the startup’s product demos (shown to Apple and Amazon, in Manohar’s account).
WO 2016/097470 A1 · US 2016/0173869 A1
Bokeh before Portrait mode. A main high-quality camera paired with auxiliary stereo cameras producing depth for defocus effects — filed two years before the iPhone 7 Plus made computational bokeh mainstream.
In plain English. One great camera takes the photo; two cheap helper cameras figure out the depth. The helpers are precisely calibrated and produce rough depth maps, while the main camera — whose lens can be swapped — shoots beautiful high-resolution photos. The trick is transferring the helpers’ depth into the main camera’s viewpoint, using the main camera’s own focus blur as an extra depth cue: a full-resolution photo plus a complete depth map, with no bulky depth hardware.
Two depth cues are better than one. Fuses stereo disparity with defocus blur — depth that holds up exactly where each cue alone fails.
Topic 03
The MIT years: quadrotors that carry, cooperate, and adapt — plus the optimal-control theory underneath.
The control theory under the drones. Optimal trajectories for redundant actuators — from his Mitsubishi research-lab years, feeding the aerial-robotics work.
In plain English. The doctoral thesis behind the drone papers: drones should graduate from “fly and sense” to jobs involving touch. It develops the full stack — how a quadrotor behaves the instant it touches something, a lightweight docking interface, controllers that switch from flying to pushing on contact, high-speed docking, teams of drones cooperating, a drone opening a door without knowing where the hinge is, and the bendable ParaFlex airframe — all validated in the STEVE simulator.
The doctoral foundation. His MIT PhD thesis on autonomous aerial manipulation — the control theory everything else stands on.
In plain English. Instead of using drones just to fly around and observe, this work puts them to physical work: fly to an object, dock onto it, and push it. The hard part is the moment of contact, so each drone’s flight controller hands off to a separate pushing controller the instant docking is detected. The paper shows teams cooperating — two drones docked to a 4 kg panel, pushing it along a commanded path — proven in simulation and lab experiments.
Aerial robotics as a crew, not a solo act. Teams of quadrotors cooperating to carry and manipulate objects — coordination as a control problem.
In plain English. Before trusting a self-adjusting flight controller on a real drone, you test it somewhere safe. The authors built STEVE, a modular simulation testbed for a four-rotor mini-helicopter in which software pieces can be swapped one by one for real hardware — gliding from pure simulation to real flight without rewriting everything. They prove it by flying a simulated quadrotor whose adaptive controller keeps working even as motors degrade or fail.
The flight simulator the lab trusted. A testbed for proving adaptive controllers in simulation before risking real hardware — infrastructure as research.
In plain English. Most drones are rigid — hit a wall and something breaks. This paper designs ParaFlex, a quadrotor whose arms join through spring-loaded joints, so the frame flexes on impact and springs back instead of shattering. It works out the math of how a bendy drone flies (the wobble changes the flight dynamics) and designs a controller that stays stable through collisions — experiments show far smaller impact forces than an equivalent rigid drone.
A bending drone doesn’t fly like a rigid one. Models flexibility in the airframe itself — control that accounts for the wobble instead of ignoring it.
Topic 04
Drones as studio lights. Robotics in service of the image — the work that made MIT News.
In plain English. Professional rim lighting — the glowing outline around a subject’s edge — is hard to get right because the light must sit in exactly the right spot. The authors mounted a flash and continuous light on a small quadrotor (nicknamed “litrobot”) that watches the photographer’s live camera feed, measures the rim width in the image twenty times a second, and flies itself to hold it steady. The photographer just dials in the rim width; the drone handles tracking the subject and staying out of frame.
Drones as studio lights. Autonomous quadrotors that position themselves around a subject for perfect photographic rim lighting — robotics in service of the image.
In plain English. The journal extension of the drone-lighting work, with full indoor experiments proving the drone holds the desired rim width while subject and photographer move — plus outdoor photo-shoot trials studying the practical ergonomics of working with a flying light. The authors’ claim: the first demonstrated system for studio-quality lighting produced by an autonomous aerial robot.
Light that chases the subject. Extends drone rim-lighting to moving subjects — the quadrotor re-plans in real time so the perfect highlight never slips.
In plain English. MIT News covered the project: researchers at MIT and Cornell built a small autonomous helicopter that positions itself to create rim lighting — the tricky effect where only the subject’s edge is strongly lit. The photographer picks the light direction and rim width through a simple interface; the drone maintains it automatically, compensating as the subject turns or the photographer moves.
The work, in the wild. MIT News covered the drone-lighting project — the moment the lab’s flying studio lights stepped into public view.
Topic 05
Touch you can believe, and input devices with room to move — from virtual walls to his earliest invention on record.
US 2012/0068927 A1 · GB 2,434,227 B (granted 2010)
His earliest invention on record. Three degrees of freedom in a single input device — and his second most-cited invention.
In plain English. A force-feedback device can only push back so hard before it starts buzzing — that ceiling is its “passive stiffness.” This paper raises the ceiling with real, physical damping instead of simulated damping: short the DC motor’s terminals through clever H-bridge wiring and the motor itself soaks up the jitter without injecting noise into the control loop. On their test device the virtual wall felt about 33% stiffer — with no extra hardware.
Stiffer virtual walls, calmer motors. Damping the motor instead of fighting it — haptic devices that feel more solid with less energy.
In plain English. Virtual walls feel mushy because the digital control loop can only update so fast before going unstable — capping how stiff a virtual surface can feel. The authors moved the haptic control loop off the CPU onto a dedicated chip (an FPGA) running hybrid continuous/discrete-time feedback, so forces update far faster. Tested head-to-head on the same device, the hybrid loop rendered much stiffer walls — and freed the main CPU for graphics and physics.
Touch you can believe. Hybrid continuous–discrete control that renders a virtual wall as genuinely solid — the haptics behind believable contact.
In plain English. A regular mouse tracks side-to-side movement but is blind to rotation — twist it and the cursor never knows. The authors’ “mushaca” (Sanskrit for “mouse”) adds that missing degree of freedom using the same tiny 1 mm optical window and cheap camera chip already in every mouse. The trick is mathematical, not hardware: fit the whole 18×18-pixel image sequence with a least-squares method across frames, squeezing out rotation estimates a hundred times finer than a pixel — twist-sensing with no new molds, lenses, or cost.
Tiny window, precise angle. Sensing angular change through a small optical window — early work on compact optical sensing, with T. Poston.
Topic 06
Planning through what can be seen — shortest paths where visibility itself is the cost function.
In plain English. Given a 3D terrain, a “visibility volume” is the region of space from which a chosen target can be seen — the flip side of the familiar viewshed idea. The paper shows how to compute these volumes and use them to optimize paths against visibility criteria: routes that keep watch over an area, or stay out of sight.
Planning through what can be seen. Treats visibility as a first-class citizen in motion planning — paths optimized inside the volume of the observable.
In plain English. How do you send drones to observe every inch of hostile terrain while staying hidden from outposts — and flying the shortest total distance? The authors compute an “exposure surface” (a ceiling the drones must stay beneath to remain unseen), then group terrain points so each drone’s cluster yields the most coverage per mile flown. Key finding: grouping by visibility first and distance second beats the reverse — cutting total path length by over 25%.
Shortest safe paths. Minimum-distance routes through hostile terrain under partial and complete visibility — planning where being seen is the cost.
In plain English. Picture a surveillance drone that must photograph every hill and valley of rugged terrain while staying hidden from eyes below. This paper takes on that trade-off directly: planning flight paths that maximize the ground the camera actually sees while minimizing exposure to hostile observers — coverage planning with stealth as a first-class constraint, in the team’s early line of work on UAVs over hostile 3D terrain.
Seen, but not seen. UAV coverage planning that maximizes area covered while preserving covertness over undulating terrain.
The full citation record — 1,100+ citations, h-index 16 — lives on the Google Scholar profile. This page is the curated record, grouped by topic; the old Scholar mirror has been retired.