Four pixels of trim, hanging in open air

There was a white peg hanging off the front bumper of tonight's car. Three pixels wide, five tall, bolted to nothing. I spent the best hour of the night on it and it was not on the bumper.

The frame it appears in. Desktop stage, 718x403. Good luck.

It is in there, off the front corner, at about x=347. The stray pixels measure 191, 182, 164 against a floor sitting at 39, 35, 28, which is why it caught my eye at all in a room lit by one lamp.

Same frame, eight times up. A lit bar of trim, floating clear of the bodywork.

The theory that fit perfectly

The car is a loft: about 120 cross sections down the length, each one a superellipse whose exponent breathes so the shoulder line and the tumblehome emerge from the sweep instead of being drawn on. Panel gaps are grooves subtracted from that surface field.

Which is a lovely way to make a flank and a completely useless way to make a face. From car.ts:

// The loft cannot make a grille. Its sections are rings in the y/z plane and
// every one of their normals lies in that plane, so nowhere on this surface
// does a normal point down the car's own length — which means a groove near
// the nose recesses sideways, into the flank, and never inward into a face
// you are looking at.
//
// So the fascia is added instead of subtracted. Probe the finished shell with
// a ray straight down -X to find where its skin actually is at a given height
// and width, then stand the parts PROUD of that skin by a centimetre or two.

That rebuild landed less than an hour before the peg showed up in a capture sweep. So the theory wrote itself: a probe ray that misses the shell has nothing to stand its part on, and the part ends up in space. Bumper corner, fascia part, floating. Case closed.

The problem with that theory is not that it is unreasonable. It is that four other theories explain the same three pixels equally well, and at 718x403 there is nothing on screen to choose between them. Magnifying the PNG harder tells me facts about the PNG. I needed the frame to tell me which triangle it drew.

Which triangle owns pixel (345, 238)

So I built a picker. Drive the page to the exact camera station, walk every triangle of every visible mesh, project each one with the live camera matrices, and ask which triangles contain the pixel.

Getting at the scene from outside the app is the only bit that needs a trick. Three dispatches its scene to a devtools hook if one exists, so I install a fake one before any script runs:

await ctx.addInitScript(() => {
  window.__caught = []
  window.__THREE_DEVTOOLS__ = { dispatchEvent: (e) => window.__caught.push(e.detail) }
})

From the caught scene, react-three-fiber hands over the camera and the canvas size, which is what makes the answer come back in screenshot coordinates rather than in NDC:

const scene = window.__caught.find((o) => o && o.type === 'Scene')
const st = scene.__r3f.root.getState()
const cam = st.camera, W = st.size.width, H = st.size.height
const V = cam.matrixWorldInverse.elements, P = cam.projectionMatrix.elements

Then the projection, by hand, because the instance transform matters: the 335 impact craters are one instanced mesh, and a picker that ignores instanceMatrix will happily blame a crater for something a wheel did.

const proj = (id) => {
  let lx = pos.getX(id), ly = pos.getY(id), lz = pos.getZ(id)
  if (im) {
    const e = im.slice(iOff * 16, iOff * 16 + 16)
    const t = mul(e, [lx, ly, lz, 1]); lx = t[0]; ly = t[1]; lz = t[2]
  }
  const w = mul(wm, [lx, ly, lz, 1])
  const v = mul(V, w)
  const e = mul(P, v)
  return { x: (e[0] / e[3] * 0.5 + 0.5) * W, y: (-e[1] / e[3] * 0.5 + 0.5) * H, d: -v[2], w }
}

Containment is three edge cross products with a bounding-box reject in front of them, and the hit gets reported with its depth, its world positions and its first vertex colour:

const s1 = (q[1].x-q[0].x)*(py-q[0].y)-(q[1].y-q[0].y)*(px-q[0].x)
const s2 = (q[2].x-q[1].x)*(py-q[1].y)-(q[2].y-q[1].y)*(px-q[1].x)
const s3 = (q[0].x-q[2].x)*(py-q[2].y)-(q[0].y-q[2].y)*(px-q[2].x)
const inside = (s1>=0&&s2>=0&&s3>=0)||(s1<=0&&s2<=0&&s3<=0)

Results filter to d > 3 metres (nothing between the camera and the exhibit is under suspicion), sort by depth, and print the nearest five. Forty lines. It is not fast, a few hundred thousand triangles projected in plain JS, but it runs once and it answers the exact question.

A Raycaster would have found the same surface. It would not have handed me the whole depth-sorted stack under the pixel, or the vertex colour, which turned out to be the entire answer. I had also already thrown three's raycaster out earlier that night: casting 335 impact trajectories at the shell took 425 ms through Raycaster and 43 ms through a typed-array Möller-Trumbore over a front-facing triangle subset, same distribution. Doing my own intersection tests was already the habit of the evening.

The answer was a colour

The picker came back with car-trim, vertex colour 0.58 / 0.55 / 0.49.

Not the fascia. The mesh name alone could not tell me more than that, because everything on this car that never moves relative to the body lives in one merged buffer with one material: bumper, grille, lamps, arches, wheels, mirrors, handles, wipers. That merge took thirty meshes down to nine draw calls, which the phone needs, and it costs exactly this: the object you picked has no name of its own any more.

The colour still does. Vertex colours in three are linear, so invert the sRGB transfer:

((201/255 + 0.055) / 1.055) ^ 2.4 = 0.58
((196/255 + 0.055) / 1.055) ^ 2.4 = 0.55
((186/255 + 0.055) / 1.055) ^ 2.4 = 0.49

c9c4ba. One grep, one hit:

const arch = new THREE.Color('#c9c4ba')

The wheel arch lip. Nothing to do with the fascia rebuild, and a quarter of a metre back from the bumper corner I had spent forty minutes staring at.

Two parametrisations of one circle, and only one knows where it stops

The wheel openings are struck as circles about each axle. So is the lip that runs around them, deliberately, so it sits on the cut instead of near it:

const WHEEL_R = 0.3
const ARCH_R = 0.35
const ARCH_Y = 0.295

But the shell is not a circle. The shell is a lofted profile, and the profile only follows that circle for as long as the arch is an arch. Here is the bottom line of the car through the front axle, each pair x then y, in metres from the centre of the car:

  [-0.92, 0.335], [0.78, 0.335], [0.8, 0.412], [0.84, 0.491], [0.89, 0.55],
  [0.97, 0.606], [1.05, 0.636], [1.13, 0.645], [1.21, 0.636], [1.29, 0.606],
  [1.37, 0.55], [1.42, 0.491], [1.46, 0.412], [1.48, 0.335], [1.66, 0.31],

Flat along the rocker at 0.335 for the whole 1.7 m between the arches, up onto the circle at 0.78, cresting at 0.645 directly over the axle at 1.13 (which is ARCH_Y + ARCH_R exactly), back down to 0.335 at 1.48, and then gone to the bumper. Those two hand-tuned x values, 0.78 and 1.48, are just 1.13 ± sqrt(0.35² − 0.04²). The profile leaves the circle at the rocker line, because that is where the arch ends.

The torus knew nothing about that. It was struck from -10.8° to 190.8°, a half turn with a bit of overlap at each end so the lip would visibly wrap past the opening. Meanwhile the lip's own circle, 4 mm proud of the arch, crosses the rocker line at

asin((0.335 - 0.295) / 0.354) = 6.49°

Everything between -10.8° and +6.49° is lip drawn where there is no arch to draw it on. That is 17.3° of arc at 0.354 m radius: 10.7 cm of tube in open air, at both ends, of all four arches. Eight of them.

Seven were invisible. They die into dark under the sill, or they cross white bodywork where a pale sliver on pale plaster reads as nothing. The eighth, the forward end of the front arch, projected past the corner of the bumper against black floor, with the room's one lamp on it. Contrast made one of eight defects visible, and contrast is also why I spent forty minutes convinced the bug was in the part of the car nearest to it.

The fix is one asin

Start the arc where the tube's own underside meets the rocker line, not where its centreline does, so the end caps land inside the bodywork instead of level with it:

const LIP_TUBE = 0.019
const LIP_START = Math.asin((0.335 - ARCH_Y + LIP_TUBE) / (ARCH_R + 0.004))

function addArchLip(trim: Trim, geometries: THREE.BufferGeometry[]): void {
  const lip = new THREE.TorusGeometry(
    ARCH_R + 0.004,
    LIP_TUBE,
    5,
    26,
    Math.PI - 2 * LIP_START,
  )

9.59° instead of 6.49°, and the arc goes from 201.6° of sweep to 160.8°. The lip lost a fifth of its length and gained the only property that mattered, which is that both ends die into something.

Same beat after the fix. The aim moved in the same pass, so the framing is not identical, but the arch is: the lip runs the opening and stops.

What I would keep

The bright end is not the broken part. My eye went to the highest-contrast pixel in the frame and my debugging followed it there. The thing that makes a defect visible and the thing that causes it are two different pieces of geometry more often than feels reasonable.

A procedural mesh has nothing to click. No outliner, no file, no object to select in a viewport. Everything is a function of a few constants, which is wonderful until you need to know which of them is wrong, at which point you have to build the instrument. Mine was forty lines and twenty minutes, and it is the cheapest hour I spent all night.

Colour-code merged geometry on purpose. Merging thirty meshes into one buffer throws away every name. Distinct vertex colours per part are half a kilobyte and they keep the buffer answerable: pick a pixel, get a colour, grep the source.

Fixing a mechanism removes defects nobody reported. One asin deleted the same sliver from the rear arches, which no one had flagged and I had never seen. When a one-line change makes bugs you did not know about disappear, that is the signal you found the cause and not the symptom.

The evidence pointed at the bumper. The instrument said the wheel arch. The instrument was right, which is more or less the argument the whole game is making.