SegWit

BIP 141: Segregated Witness

The Segregated Witness (SegWit) upgrade in 2017 changed the structure of transaction data in Bitcoin.

The main reason for this upgrade was to fix transaction malleability (I'll explain this in a moment). The other significant change included a block size increase.

What was the main change?

Legacy transaction

In a legacy transaction, the unlocking code (and signatures) sit next to each input, so the unlocking code is spread throughout the transaction data.

The TXID is then created from the entire transaction data:

SegWit transaction

In a segwit transaction however, all of the unlocking code (and signatures) are moved to the end of the transaction data instead.

The TXID is then created from all of the transaction data, except for the unlocking code:

As a result, the TXID in a segwit transaction is only influenced by the effects of the transaction (the movement of bitcoins), and not by the code required to validate the transaction (i.e. the signatures required for unlocking the bitcoins for spending).

So in essence, you have separated the "validating" part (unlocking code) from the rest of the transaction.

If you were to refer to this validation code as witness data (as a cryptographer might), you could say that you have "segregated the witness". *wink*

What are the benefits?

1. Fixes transaction malleability
2. Increased block capacity

1. Fixes transaction malleability

In Bitcoin, transaction malleability refers to the fact that the TXID of a transaction can be changed by altering the signatures:

A signature can be altered by inverting the s value. The signature is still valid and the transaction has the same effect, but the TXID is different.

This means that when you send a legacy transaction in to the network, any node has the ability to change the TXID before passing it on:

Eventually your transaction will make it in to the blockchain under a different TXID than you expected, which would be somewhat annoying.

However, if the signatures are no longer part of the TXID, it's no longer possible for someone else to change the TXID of your transaction later on:

So in other words, SegWit makes TXIDs reliable.

2. Increased block capacity

Due to the fact that the unlocking code was moved to a new witness field in the transaction data, the way block sizes were calculated could also be changed.

Previously, transactions were measured in bytes, and the block size limit was 1,000,000 bytes (1 MB):

With SegWit, transactions are no longer measured in bytes. Instead, transactions and blocks were given a new metric called weight:

• A block has a maximum size of 4,000,000 weight units.
  • A normal byte in a transaction is equal to 4 weight units.
  • A witness byte in a transaction is equal to 1 weight unit.

So basically, the block size limit is multiplied by 4 to give you the new block weight limit.

Each byte in a transaction is then also multiplied by 4 to give you a transaction weight. However, you only multiply the bytes of witness data by 1, which basically gives you a 75% discount on how much space the "unlocking code" takes up in a block.

So you could say that unlocking data takes up a quarter of the space it used to, which means there's more space in the block overall for transaction data.

Was SegWit a block size increase?

Yes, each block can now be greater than 1,000,000 bytes (1 MB) in size.

So whilst the block size limit was not increased by a specific number of bytes, the discounted weight for unlocking data means that blocks can exceed the previous 1 MB limit.

How much of a block size increase was SegWit?

The SegWit upgrade increased the maximum size for typical blocks to around 1.8 MB.

You see, a typical block of transactions consists of around 60% unlocking data¹. So if we calculate the weight of a 1 MB block consisting of "typical" transaction data, we get:

400,000 bytes * 4 = 1,600,000 weight units
600,000 bytes * 1 =   600,000 weight units

Total Weight      = 2,200,000 weight units

Now, if a block can now weigh a maximum of 4,000,000 weight units, we can work out how much of an increase this gives us:

4,000,000 / 2,200,000 = 1.81

So you could say this was effective block size limit increase to 1.8 MB.

1. I got this 60% figure by running through blk.dat files and adding up the scriptSig data for all the transactions in a block, and comparing it to the total size of the block. I haven't done an exhaustive test, but 60% seems like a fair average. For example, here are the result for blk00700.dat.

Why were the changes implemented in this way?

Or, to put it another way…

If you wanted to fix transaction malleability and increase block capacity, surely there's a more straightforward way of doing it? Why do you need to restructure the transaction data, and create a new metric called "block weight"?

Good question. And you're right – these changes could have been made much more simply. For example, you could have just done this:

However, if you did this, the transactions and blocks would become "invalid" under the current rules.

Basically, this means nodes on the network would reject these new transactions and blocks, because they do not comply with their rules on how transactions and blocks should "look".

For example, one of the rules was that each block must be 1 MB or less.

Therefore, if you wanted to make these changes, you would need to make everyone on the network upgrade their software (and obviously agree with the changes).

Because if you didn't, you would end up with a network that builds two different blockchains – upgraded nodes building a blockchain using the new rules, and old nodes continuing to build a blockchain using the old rules.

This in known as a hard fork. It can work, but it's risky, and will cause problems for those who do not upgrade.

How did SegWit avoid a hard fork?

Instead of SegWit being a hard fork, it was implemented as a soft fork.

With the SegWit upgrade, the transactions and blocks still follow the current rules of the bitcoin network, so all nodes still see SegWit blocks as valid. Therefore, "old" nodes will accept these "new" blocks and add them to their blockchains too.

Therefore, old nodes still keep up with the new nodes, even if they do not upgrade.

Everyone stays in sync with a single version of the blockchain with a soft fork.

The downside for "old" nodes is that they cannot take advantages of the new features of SegWit until they upgrade. However, until then they can continue to make "old-style" transactions as normal and keep up with the blockchain.

So in summary, the SegWit upgrade may seem like a "hacky" way of fixing transaction malleability and increasing block capacity, but this approach avoids the problem of trying to get everyone to upgrade to the new software (or else get left behind).

When was SegWit activated?

Segwit was activated on 24th August 2017, 01:57:37, at block height 481,824.

This was when nodes started enforcing the new consensus rules of the SegWit upgrade for all new blocks and transactions.

How did SegWit come in to effect?

The Segregated Witness upgrade came in to effect when 95% of miners signalled readiness for it.

Miners can signal readiness by using a designated version number in the blocks they mine.

The version field is part of the block header.

So when 95% of blocks had this version number, SegWit was scheduled for activation:

The 95% threshold is calculated within a target readjustment period. If the 95% threshold is met, the soft fork is activated at the start of the next target adjustment period (which is 2016 blocks, or roughly 2 weeks).

Was there a time limit on activation?

Yes, this was the activation window:

If not enough miners signalled readiness for the Segregated Witness upgrade by midnight 15th November 2017, the proposal would have expired.

Activation timeline

Here's a table showing the number of blocks signalling for SegWit across each target period leading up to the activation:

As you can see, the 95% threshold of miners signalling readiness for SegWit was exceeded during the target adjustment period between blocks 477,792 and 479,807.

Consequently, the SegWit upgrade was activated 2,016 blocks (roughly 2 weeks) later at block 481,824:

Why were miners given the decision on activation?

Because if you want a soft fork to be successful, you want a majority of miners mining the "new" type of blocks on to the blockchain.

This is so the blockchain with "new" blocks on it will outpace any blockchain being built with "old" blocks (from any non-upgraded miners who could still be mining).

As a result, the "new" blockchain will be built faster than any blockchain being built with "old" blocks, so all nodes will naturally adopt the same longest chain:

Having a majority of mining power keeps everyone on the network on the same chain.

So having a strong majority of miners on board allows for a smooth upgrade, as it ensures that the whole network will converge on one single blockchain.

It's not necessarily that miners are the most knowledgeable group for deciding upon the merits of a soft fork upgrade; it's more that they're needed to ensure a smooth upgrade for the entire network.

What happens if I don't run the SegWit upgrade?

If you're running an old node (e.g. Bitcoin Core v0.13.0 or below), any SegWit nodes that you are connected to will strip out all of the witness data from transactions before sending them to you.

What this means is:

• You will still receive the same transactions as everyone else.
• If you receive a SegWit transaction, you will see the movement of bitcoins, but you will not see any of the unlocking code data.

So basically, your node will get a "lightweight" version of SegWit transactions.

How do I upgrade?

That's the spirit.

• Bitcoin Core – Just make sure you're using version 0.13.1 or greater.
• Other Wallets – Almost all modern wallets these days support SegWit transactions.

SegWit has been around for so long now that it's unlikely that you'll run in to any software that does not support it (unless it's obviously old).

Resources

• BIP 141 (Consensus Layer)
• BIP 144 (Peer Services)
• Bitcoin.org – Segregated Witness Benefits

Thanks

• Pieter Wuille – for explaining SegWit transaction data structure (amongst other things).
• Gregory Maxwell and Luke-jr – for explaining block weight.

Further reading

• Segregated Witness Explained [Video] – A good video explanation of SegWit, with helpful visualizations of the change in transaction data structure.