BIP 141: Segregated Witness (21 December 2015)

What is Segregated Witness?

Segregated Witness is a proposal to change the structure of bitcoin transaction data.

The main reason for doing this is to fix the problem of transaction malleability (don’t worry, I’ll explain this in a moment). However, this change also allows for other benefits, such as increasing the number of transactions that can fit in to a block.

What are the changes?

In the current transaction data structure, signatures (the code that “unlocks” existing bitcoins) sit next to each input, so this unlocking code is spread throughout the transaction data. The TXID is then created from the entire transaction data.

In Segregated Witness (occasionally and affectionately referred to as SegWit from here on out), the proposal is to move all of the unlocking code to the end of the transaction data. The TXID is then created from all of the transaction data, except for the unlocking code.

Result:

The TXID is only influenced by the effects of a transaction (the movement of bitcoins), and not by any code needed to validate the transaction (i.e. signatures used to unlock existing bitcoins so that they can be spent).

So in essence, you have separated the “validating” part (unlocking code) from the “effective” part of the transaction.

If you were to refer to this validation code as witness data (as a cryptographer might), you could say that you have “segregated the witness”. *wink*

1. Fixes Transaction Malleability

In Bitcoin, Transaction Malleability refers to the fact that the TXID of a transaction can be changed by altering the unlocking code in the transaction.

This means that when you send your transaction in to the network, any node has the ability to change the the TXID before passing it on.

Eventually your transaction will make it in to the blockchain under a different TXID than you expected, which would be somewhat annoying.

However, if the unlocking code is no longer part of the TXID, then no node will have the ability to change the TXID of your transaction.

So in other words, Segregated Witness makes your TXIDs reliable.

2. Increases Block Capacity

Due to the fact that the unlocking code has been moved to a new “witness” field in the transaction data, this means the way block sizes are calculated can also be changed.

Currently, the block size limit is 1,000,000 bytes (1MB). Transactions are measured in bytes.

With Segregated Witness, the block size limit is no longer measured in bytes. Instead, blocks and transactions are given a new metric called “weight”.

A block has a maximum weight of 4,000,000.
- A typical byte in a transaction weighs 4.
- A witness byte in a transaction weighs 1.

So basically, the block size limit is multiplied by 4 to give you the new block weight limit. Each byte in a transaction is then also multiplied by 4 to give you a transaction weight. However, you only multiply the bytes of witness data by 1, which basically gives you a 75% discount on how much space it takes up in a block.

So you could say that unlocking data takes up 1/4 of the space it used to, which frees up more space in the block for transactions.

Is this a block size increase?

Yes, each block can now be greater than 1,000,000 bytes (1MB) in size.

So whilst the block size limit has not been increased to a specific number of bytes, the discounted weight for unlocking data means that blocks will typically exceed the current 1MB limit.

How much of a “block size increase” is Segregated Witness?

1.8MB, roughly.

Where did I get this figure from?

A typical block of transactions currently consists of around 60% unlocking data. ¹

So if we calculate the weight of a 1MB block consisting of a “typical” amount of locking data, we get:

400,000 bytes * 4 = 1,600,000 weight units
600,000 bytes * 1 =   600,000 weight units

Total Weight      = 2,200,000 weight units

Now, if a block can weigh a maximum of 4,000,000 weight units, we can work out how much of an increase this gives us:

4,000,000 / 2,200,000 = 1.81

So you could say that this is an effective block size limit increase to 1.8MB.

Just because the block size limit is changing from 1,000,000 bytes to 4,000,000 weight units, it does not make Segregated Witness an effective block size increase to 4MB, because a typical block is not going to be filled exclusively with witness data (1 weight unit per byte).

Instead, transactions will contain a combination of normal data (4 weight units) and witness data (1 weight unit). So the “block size increase” will vary depending on the composition of transactions in a block.

Why is it being implemented in this way?

Or, to put it another way…

If you want to fix transaction malleability and increase block capacity, surely there’s a more straightforward way of doing it? Why do you need to restructure the transaction data, and create a new metric called “block weight”?

Good question. And you’re right – these changes could be made much more simply. For example, you could just do this:

However, if you did this, the transactions and blocks would become “invalid” under the current rules.

Basically, this means nodes on the network would reject these new transactions and blocks, because they do not comply with their rules on how transactions and blocks should “look”.

For example, one of the rules is that each block must be 1MB or less.

Therefore, if you wanted to make these changes, you would need to make everyone on the network upgrade their software (and obviously agree with the changes).

Because if you didn’t, you would end up with a network that builds two different blockchains – new nodes building a blockchain with the new blocks, and old nodes continuing to build a blockchain with old blocks.

This in known as a hardfork. It can work, but it’s risky, and can cause problems for those who do not upgrade.

How does Segregated Witness avoid this problem?

With the Segregated Witness approach, the transactions and blocks still follow the current rules of the bitcoin network, so all nodes will still see SegWit blocks as valid. Therefore, “old” nodes will accept these “new” blocks and add them to their blockchains too.

Therefore, old nodes will keep up with the new nodes, even if they do not upgrade.

Everyone stays in sync with a single version of the blockchain.

The downside for “old” nodes is that they cannot take advantages of the new features of SegWit until they upgrade. However, until then they can continue to make “old-style” transactions as normal.

In summary, Segregated Witness may seem like a round-the-houses way of fixing transaction malleability and increasing block capacity, but this approach avoids the problems of trying to get everyone to upgrade to the new software (or else get left behind).

When will this come in to effect?

Segregated Witness will come in to effect when 95% of miners signal that they are ready for it.

Miners can signal readiness by using a designated version number in the blocks they are mining.

The version field is part of the block header.

When 95% of blocks have this version number, SegWit will be scheduled for activation:

The 95% threshold is calculated between two re-target blocks. If 95% is exceeded, the soft fork is activated at the next retarget block (2016 blocks later, which is roughly 2 weeks).

Why are miners being given the decision on activation?

Because if you want a soft fork to be successful, you want a majority of miners mining the “new” type of blocks on to the blockchain.

This is so that the blockchain with “new” blocks on it will outpace a blockchain with “old” blocks (from any non-upgraded miners who could still be mining). As a result, the “new” blockchain will be built faster than any blockchain being built with “old” blocks, so all nodes will naturally adopt the single longest chain.

Having a majority of mining power keeps everyone on the network on the same chain.

In summary, having a strong majority of miners on board allows for a smooth upgrade, as it ensures that the whole network will converge on one single blockchain.

It’s not necessarily that miners are the most appropriate group for deciding on the merits of Segregated Witness; it’s more that they’re needed to ensure a smooth upgrade for the entire network.

Is there a time limit on deployment?

Yes.

Start: Midnight 15th November 2016
End: Midnight 15th November 2017

If Segregated Witness is not activated by the Midnight 15th November 2017, the proposal will expire.

What happens if I don’t upgrade my node?

If you’re running an old node, any SegWit nodes that you are connected to will strip out all of the segregated witness data from transactions before sending them to you.

What this means is:

You will still receive the same transactions as everyone else.
If you receive a SegWit transaction, you will see the movement of bitcoins, but you will not see any of the validation data.

So basically, your node will get a “lightweight” version of SegWit transactions.

How do I upgrade?

That’s the spirit.

Bitcoin Core - Just make sure you’re using version 0.13.1 or greater.
Other Wallets - If you’re using a different wallet, you can see if it’s ready by checking this Segregated Witness adoption list. (I like Electrum)

Resources

Thanks

Pieter Wuille; for explaining SegWit transaction data structure (amongst other things).
Gregory Maxwell + Luke-jr; for explaining block weight.