Why your video plays sideways, and why fixing it takes half a second

The short version

Your phone’s camera sensor is fixed in the body in landscape orientation. When you film holding the phone upright, the sensor still records a landscape frame; the phone just writes a flag into the file saying “display this rotated 90°”. Every player on the phone honours the flag. Some software elsewhere — older editors, some web players, certain upload pipelines — ignores it or strips it, and the result is a video that plays on its side. Putting the flag back is a metadata edit: it takes a fraction of a second regardless of file size and does not re-encode anything.

What the flag actually is

An MP4 or MOV file is a tree of boxes. The one describing a video track’s presentation is called tkhd (track header), and it contains a 3 × 3 transformation matrix — nine numbers, 36 bytes — that a player applies to every frame before drawing it. For an unrotated video the matrix is the identity. For a video shot upright on a phone it is a 90° rotation. The pixels in the file are landscape; the matrix says “turn them”. Android writes the same information as a separate rotation value; ffmpeg reports both as rotate: 90 in the stream metadata.

This design is deliberate. Rotating the actual pixels while recording would cost a full extra pass through the encoder on every frame, on a battery. Writing nine numbers costs nothing. It is the same reasoning behind EXIF orientation in photos, and it fails in the same way: any consumer that does not read the tag shows the raw orientation.

Who drops it

• Older desktop editors and players written before phones shot vertical video. VLC and QuickTime have honoured the matrix for years; some corporate video systems and older Windows Media Player builds do not.
• Transcoders that copy the video stream but rebuild the container. A tool that says “no re-encoding, lossless” may still write a fresh tkhd with an identity matrix. The pixels survive; the instruction does not.
• Some web uploads. A site that re-muxes on upload can lose the tag; a site that re-encodes usually bakes the rotation in correctly.
• Concatenation. Joining a portrait clip and a landscape clip into one file leaves the combined track with one matrix; one of the two halves will be wrong.

Two fixes, one of them free

Rewrite the matrix. Set the tkhd matrix (or the rotation field) to the value that makes the video display correctly. It is a write of a few dozen bytes into a file that may be gigabytes; the video and audio streams are copied through untouched. Half a second, no quality loss, same file size. This is what rotate video does by default: it reads the file on your device, writes the corrected header, and hands back the file. Because nothing is decoded, it works on a 4 GB recording as fast as on a 4 MB one.

Re-encode with the rotation baked in. Decode every frame, rotate the pixels, encode again. Necessary only when the consumer is known to ignore the matrix and you cannot change the consumer — a digital signage player, a legacy CMS. It takes as long as the video is (or longer without a hardware encoder), and it is a generation of quality loss. Rotate video offers this as the second option, using the browser’s WebCodecs encoder, and says so in the interface because it is the expensive path.

Why the “free” fix sometimes doesn’t work

If the video plays sideways in one program and correctly in others, that program ignores the matrix and rewriting it will not help — only baking the rotation in will. If it plays sideways everywhere, the matrix was lost, and rewriting it is the fix. A quick test: open the file in a current browser (Chrome, Safari, Firefox all honour the tag). Sideways in the browser → matrix is wrong or missing → rewrite it. Correct in the browser, sideways in the other program → the program is the problem → bake it in.

Related traps

• Thumbnails. A frame grabbed from a sideways video comes out sideways too, because the image is taken from the stream, not the display. Fix the video first, then grab the frame.
• Trimming. Trimming without re-encoding preserves the matrix; trimming through a tool that re-muxes may not. If a trimmed clip comes back lying down, the trimmer dropped the tag, and a matrix rewrite puts it back.
• Mirrored front-camera video. That is a different flag (a horizontal flip in the same matrix) and the same fix applies.