What was the Segregated Witness upgrade?

BIP 141: Segregated Witness

The Segregated Witness (SegWit) upgrade in 2017 changed the structure of transaction data in Bitcoin.

The main reason for this upgrade was to fix transaction malleability (I'll explain this in a moment). The other significant change included a block size increase.

What was the main change?

Legacy transaction

In a legacy transaction, the unlocking code (and signatures) sit next to each input, so the unlocking code is spread throughout the transaction data.

The TXID is then created from the entire transaction data:

Diagram showing the position of signatures in a legacy transaction and the TXID being created from the entire transaction data.

SegWit transaction

In a segwit transaction however, all of the unlocking code (and signatures) are moved to the end of the transaction data instead.

The TXID is then created from all of the transaction data, except for the unlocking code:

Diagram showing the position of signatures in a segwit transaction and the TXID being created from all of the transaction data excluding the signatures.

As a result, the TXID in a segwit transaction is only influenced by the effects of the transaction (the movement of bitcoins), and not by the code required to validate the transaction (i.e. the signatures required for unlocking the bitcoins for spending).

Diagram highlighting how the unlocking code is no longer included as part of the TXID in a segwit transaction.

So in essence, you have separated the "validating" part (unlocking code) from the rest of the transaction.

If you were to refer to this validation code as witness data (as a cryptographer might), you could say that you have "segregated the witness". *wink*

What are the benefits?

  1. Fixes transaction malleability
  2. Increased block capacity

1. Fixes transaction malleability

In Bitcoin, transaction malleability refers to the fact that the TXID of a transaction can be changed by altering the signatures:

Diagram showing the TXID of a legacy transaction being changed by altering the signatures inside the transaction data.

A signature can be altered by inverting the s value. The signature is still valid and the transaction has the same effect, but the TXID is different.

This means that when you send a legacy transaction in to the network, any node has the ability to change the TXID before passing it on:

Diagram showing the TXID of a legacy transaction being modified after the transaction has been sent in to the bitcoin network.

Eventually your transaction will make it in to the blockchain under a different TXID than you expected, which would be somewhat annoying.

However, if the signatures are no longer part of the TXID, it's no longer possible for someone else to change the TXID of your transaction later on:

Diagram showing the TXID of a segwit transaction remaining the same after being sent in to the bitcoin network.

So in other words, SegWit makes TXIDs reliable.

2. Increased block capacity

Due to the fact that the unlocking code was moved to a new witness field in the transaction data, the way block sizes were calculated could also be changed.

Previously, transactions were measured in bytes, and the block size limit was 1,000,000 bytes (1 MB):

Diagram showing transactions and block capacity measured in bytes.

With SegWit, transactions are no longer measured in bytes. Instead, transactions and blocks were given a new metric called weight:

Diagram showing transactions and block capacity measured in weight units.

So basically, the block size limit is multiplied by 4 to give you the new block weight limit.

Each byte in a transaction is then also multiplied by 4 to give you a transaction weight. However, you only multiply the bytes of witness data by 1, which basically gives you a 75% discount on how much space the "unlocking code" takes up in a block.

So you could say that unlocking data takes up a quarter of the space it used to, which means there's more space in the block overall for transaction data.

Was SegWit a block size increase?

Yes, each block can now be greater than 1,000,000 bytes (1 MB) in size.

So whilst the block size limit was not increased by a specific number of bytes, the discounted weight for unlocking data means that blocks can exceed the previous 1 MB limit.

Just because the block size changed from 1,000,000 bytes to 4,000,000 weight units, it does not make SegWit an absolute block size increase to 4 MB.

This is because a typical block is not going to be filled exclusively with witness data (1 weight unit per byte). Instead, transactions contain a combination of normal data (4 weight units) and witness data (1 weight unit). So the "block size increase" varies depending on the composition of transactions in a block.

How much of a block size increase was SegWit?

The SegWit upgrade increased the maximum size for typical blocks to around 1.8 MB.

You see, a typical block of transactions consists of around 60% unlocking data1. So if we calculate the weight of a 1 MB block consisting of "typical" transaction data, we get:

400,000 bytes * 4 = 1,600,000 weight units
600,000 bytes * 1 =   600,000 weight units

Total Weight      = 2,200,000 weight units

Now, if a block can now weigh a maximum of 4,000,000 weight units, we can work out how much of an increase this gives us:

4,000,000 / 2,200,000 = 1.81

So you could say this was effective block size limit increase to 1.8 MB.

1. I got this 60% figure by running through blk.dat files and adding up the scriptSig data for all the transactions in a block, and comparing it to the total size of the block. I haven't done an exhaustive test, but 60% seems like a fair average. For example, here are the result for blk00700.dat.

Why were the changes implemented in this way?

Or, to put it another way…

If you wanted to fix transaction malleability and increase block capacity, surely there's a more straightforward way of doing it? Why do you need to restructure the transaction data, and create a new metric called "block weight"?

Good question. And you're right – these changes could have been made much more simply. For example, you could have just done this:

Diagram showing the unlocking code being ignored in a legacy transaction when creating a TXID, and a direct block size increase to 2MB.

However, if you did this, the transactions and blocks would become "invalid" under the current rules.

Basically, this means nodes on the network would reject these new transactions and blocks, because they do not comply with their rules on how transactions and blocks should "look".

Diagram showing existing nodes on the network rejecting blocks using the new hard-forking changes.

For example, one of the rules was that each block must be 1 MB or less.

Therefore, if you wanted to make these changes, you would need to make everyone on the network upgrade their software (and obviously agree with the changes).

Because if you didn't, you would end up with a network that builds two different blockchains – upgraded nodes building a blockchain using the new rules, and old nodes continuing to build a blockchain using the old rules.

Diagram showing the blockchain being split in to two separate versions due to a hard-forking change.

This in known as a hard fork. It can work, but it's risky, and will cause problems for those who do not upgrade.

How did SegWit avoid a hard fork?

Instead of SegWit being a hard fork, it was implemented as a soft fork.

With the SegWit upgrade, the transactions and blocks still follow the current rules of the bitcoin network, so all nodes still see SegWit blocks as valid. Therefore, "old" nodes will accept these "new" blocks and add them to their blockchains too.

Diagram showing old nodes who haven't upgraded accepting the new SegWit blocks and transactions.

Therefore, old nodes still keep up with the new nodes, even if they do not upgrade.

Diagram showing both old and upgraded nodes keeping up with the same version of the blockchain after a soft-forking change.

Everyone stays in sync with a single version of the blockchain with a soft fork.

The downside for "old" nodes is that they cannot take advantages of the new features of SegWit until they upgrade. However, until then they can continue to make "old-style" transactions as normal and keep up with the blockchain.

So in summary, the SegWit upgrade may seem like a "hacky" way of fixing transaction malleability and increasing block capacity, but this approach avoids the problem of trying to get everyone to upgrade to the new software (or else get left behind).

When was SegWit activated?

Segwit was activated on , at block height 481,824.

This was when nodes started enforcing the new consensus rules of the SegWit upgrade for all new blocks and transactions.

How did SegWit come in to effect?

The Segregated Witness upgrade came in to effect when 95% of miners signalled readiness for it.

Miners can signal readiness by using a designated version number in the blocks they mine.

Diagram showing the version field in the block header.

The version field is part of the block header.

So when 95% of blocks had this version number, SegWit was scheduled for activation:

Diagram showing how the SegWit upgrade was scheduled for the next retarget period after a successful signalling period.

The 95% threshold is calculated within a target readjustment period. If the 95% threshold is met, the soft fork is activated at the start of the next target adjustment period (which is 2016 blocks, or roughly 2 weeks).

Was there a time limit on activation?

Yes, this was the activation window:

Start Time:

If not enough miners signalled readiness for the Segregated Witness upgrade by midnight 15th November 2017, the proposal would have expired.

Activation timeline

Here's a table showing the number of blocks signalling for SegWit across each target period leading up to the activation:

Start Time Target Period Signalling Blocks Percentage
439,488 to 441,503 451/2016 22.37%
441,504 to 443,519 487/2016 24.16%
443,520 to 445,535 520/2016 25.79%
445,536 to 447,551 521/2016 25.84%
447,552 to 449,567 489/2016 24.26%
449,568 to 451,583 468/2016 23.21%
451,584 to 453,599 485/2016 24.06%
453,600 to 455,615 537/2016 26.64%
455,616 to 457,631 532/2016 26.39%
457,632 to 459,647 582/2016 28.87%
459,648 to 461,663 614/2016 30.46%
461,664 to 463,679 671/2016 33.28%
463,680 to 465,695 698/2016 34.62%
465,696 to 467,711 663/2016 32.89%
467,712 to 469,727 622/2016 30.85%
469,728 to 471,743 642/2016 31.85%
471,744 to 473,759 825/2016 40.92%
473,760 to 475,775 917/2016 45.49%
475,776 to 477,791 1440/2016 71.43%
477,792 to 479,807 2016/2016 100.00%

As you can see, the 95% threshold of miners signalling readiness for SegWit was exceeded during the target adjustment period between blocks 477,792 and 479,807.

Consequently, the SegWit upgrade was activated 2,016 blocks (roughly 2 weeks) later at block 481,824:

Start Time Target Period Note
479,808 to 481,823 Segwit Locked In
481,824 onwards SegWit Activated

Why were miners given the decision on activation?

Because if you want a soft fork to be successful, you want a majority of miners mining the "new" type of blocks on to the blockchain.

This is so the blockchain with "new" blocks on it will outpace any blockchain being built with "old" blocks (from any non-upgraded miners who could still be mining).

As a result, the "new" blockchain will be built faster than any blockchain being built with "old" blocks, so all nodes will naturally adopt the same longest chain:

Diagram showing a majority of miners building the longest chain with new blocks after the SegWit upgrade.

Having a majority of mining power keeps everyone on the network on the same chain.

So having a strong majority of miners on board allows for a smooth upgrade, as it ensures that the whole network will converge on one single blockchain.

It's not necessarily that miners are the most knowledgeable group for deciding upon the merits of a soft fork upgrade; it's more that they're needed to ensure a smooth upgrade for the entire network.

What happens if I don't run the SegWit upgrade?

If you're running an old node (e.g. Bitcoin Core v0.13.0 or below), any SegWit nodes that you are connected to will strip out all of the witness data from transactions before sending them to you.

Diagram showing an old node not receiving the witness data from segwit transactions when connected to an upgraded node.

What this means is:

So basically, your node will get a "lightweight" version of SegWit transactions.

How do I upgrade?

That's the spirit.

SegWit has been around for so long now that it's unlikely that you'll run in to any software that does not support it (unless it's obviously old).



Further reading