Locks
What prevents bitcoins from being stolen?
Every output in a transaction has a lock on it. This lock is a set of requirements that must be met to spend the output in a future transaction.
So in other words, these locks prevent bitcoins from being stolen (i.e. someone else spending your bitcoins), as every output we receive is encumbered by a lock.
For example, a typical lock reads something like this:

When do locks get placed on outputs?
As we know, a transaction takes existing outputs and creates new ones from them:

And it's during the creation of these outputs that we give each one their own "lock":

So when we want to send bitcoins to a friend, we create a new output, and add a lock that says "only the owner of the address 1friend1234567890... can use this output":

As a result, this new output will effectively "belong" to our friend, because they are the only person who has the private key required to unlock the bitcoins locked to this address, so nobody else will be able to spend it.
What does a lock look like?
Locks are written in a basic programming language called Script.
It's a bit tricky to explain the workings of an entire programming language in one diagram, but here we go:

This is a simplified example of a locking script; it's not exactly what Script looks like.
Now, the most interesting part of this locking script is the CHECKPRIVATEKEY
part, which is a function that we use to help set the requirements for the lock.
So for this particular output, we've set a lock that wants to compare the address 1EUXSxuUVy2PC5enGXR1a3yxbEjNWMHuem with a private key.
If we can provide this lock with the correct private key (which the owner of the address keeps secret), we can unlock it and spend it in a transaction.
How do you unlock a lock?
When you construct the transaction data, you include an "unlocking script" alongside each output you want to spend:

So for example, to unlock a typical locking script (e.g. [address] CHECKPRIVATEKEY
), we need to prove that we own the address inside the lock. To do this, we provide the private key connected to the address.

So when a node receives this transaction data, they will run the "locking"+"unlocking" scripts together to see if your private key is mathematically connected to the address.

If everything is cool, the node accepts the transaction and passes it on to other nodes, who will each in turn run the "locking"+"unlocking" script before accepting the transaction.
And that's how you unlock a lock on an output.
Aren't we giving away our private key?
Astute observation.
Confession: We don't actually put our private key into the unlocking script.
You see, to save us from giving our private key away within the transaction data, we use the private key to create something called a digital signature instead:

Obviously I lied about that CHECKPRIVATEKEY
function as well.
However, there is a function that compares an address with a digital signature, and it's called CHECKSIG
:

And thanks to the mathematics of digital signatures and the CHECKSIG
function, we can still lock outputs to addresses, and unlock them without having to give away the private key.
Awesome.
There are many different functions available in the Script programming language. The CHECKSIG
function is designed for locking an output to a specific address, but you can use others (and in various combinations) to create much more complex locks.
For example, you could create a lock that can only be unlocked after a specific date, or a lock that can only be unlocked by the owners of two (or more) different addresses.
This is why bitcoin is sometimes referred to as "programmable money".