Protocol Update 7 - Smart contract costs and new queries, stake cooldown changes, disabling shielded transfers, revised block hashing¶
Effective on Testnet |
planned Sep 30, 2024 |
|---|---|
Effective on Mainnet |
planned Oct 30, 2024 |
Specification hash |
|
Specification¶
Protocol update
Protocol Version: 7
Builds on Protocol Version: 6
Abstract
This document describes the changes in Concordium protocol version 7 compared to
protocol version 6.
The protocol change has the following main effects
- redefines smart contract costs, reducing the cost of smart contract calls;
- adds support for querying the module reference and contract name of a smart
contract instance in smart contracts;
- changes cooldown behavior so that:
* when validators and delegators reduce their stake, the reduction is
immediately effective for future stake calculations, and the amount of the
reduction is locked for a cooldown period;
* validators and delegators can make further changes to their stake while
they already have stake in cooldown;
- disallows shielded (or encrypted) transfer transactions (`TransferToEncrypted`,
`EncryptedAmountTransfer` and `EncryptedAmountTransferWithMemo`).
- redefines the block hashing scheme to better support light clients.
Background
Smart contract execution costs are reduced in protocol version 7 because of
efficiency improvements in the implementation of smart contract execution.
The revised cost model is based on an execution model where the interpreter
stack is compiled to registers.
The new protocol version also imposes a limit on the nesting depth of
inter-contract calls to 384. This is since each live execution frame can
require up to 32MB of memory, so limiting the number of active execution
frames in this way safeguards against excessive memory use.
* * *
As of protocol version 7, smart contracts are able to query the module
reference and contract name of a smart contract instance. This can be used
to identify the code that is behind a smart contract instance, for example as
a means of authenticating a smart contract before interacting with it. A
particular motivation for this is to implement a factory pattern, where one
smart contract (the factory) is responsible for initializing instances of
another (the product). On Concordium, one smart contract cannot directly
create instances of another, but it can be used to invoke an initialization
method on another. The additional queries allow the factory to ensure that
the contract instance it is initializing is of the correct type.
* * *
Prior to protocol version 7, when a validator or a delegator reduces or
removes its stake, the amount of the stake reduction is held in a cooldown
state. During the cooldown period, the stake cannot be spent, and remains
part of the effective stake (used for lottery and reward calculation). Once
the cooldown period has elapsed, the stake is available to be spent, and
no longer contributes to the effective stake of the validator or delegator.
While the cooldown is in effect on an account, no other changes can be made
to the account's stake.
From protocol version 7, when a validator or delegator reduces its stake,
the stake reduction takes effect straight away, but the amount of the
reduction enters a cooldown queue. (Note that "straight away" here does
not mean that it has an instant effect on the lottery and rewards. The
effective stakes of validators and delegators are sampled in the first block
of the last epoch before each payday, with the sampled stakes being used
for the upcoming payday. Although the change is recorded on the account
immediately, it will be between 1 epoch and 1 payday plus 1 epoch until
the change affects the proof-of-stake lottery used for consensus.)
Stake in the cooldown queue is considered to be inactive stake, and, as
such, does not contribute to the lottery calculation. However, the
inactive stake still cannot be spent, although it is possible to transfer
it back to active stake.
When stake is removed from a validator or delegator, it enters cooldown
at the point that the removed stake is no longer part of the effective
stake for lottery calculations. Specifically, when the stake is removed,
it first enters a "pre-pre-cooldown" state. When the stake snapshot is
taken for the next payday (in the first block of the epoch before the payday)
it is moved from "pre-pre-cooldown" to "pre-cooldown". At the subsequent
payday, it is moved from "pre-cooldown" into "cooldown", and the cooldown
expiry time is set based on the cooldown time chain parameter. In
particular, the cooldown officially starts at this payday. At the first
payday at or after the cooldown expiry time, the stake finally exits
"cooldown" and becomes available to spend. An account's active stake
can be reduced while there is still inactive stake in cooldown, leading
to multiple amounts in cooldown (or pre-cooldown or pre-pre-cooldown)
that expire at different times.
If the active stake on an account is increased, the increase is first
transferred from the inactive stake in preference to taking from the
unstaked balance. Specifically, it is taken first from "pre-pre-cooldown",
then "pre-cooldown", and then from the "cooldown" stakes in decreasing
order of expiry time. An account can also switch freely between being
a validator and being a delegator without its stake entering cooldown.
(Prior to protocol version 7, it was necessary to remove the validator
or delegator on an account, waiting out the cooldown period, before
instating a delegator or validator.) As a consequence of this, from
protocol version 7, no distinction is drawn between the cooldown time
for validators and for delegators: stake entering cooldown expires
after the minimum of both times.
The purpose of these changes is to maximise the flexibility of staking,
while ensuring that any stake must still be locked for the cooldown
period before it can be spent.
* * *
Since the initial protocol version, the Concordium blockchain has
supported shielded transfers (also known as encrypted transfers).
This mechanism allows an account to "shield" part of its balance,
cryptographically encrypting it so that the amount cannot be inspected
except by the owner of the account (using its secret encryption key).
(It is possible for a sufficient combination of identity disclosure
authorities (e.g. 2 of 3) to act in concert with the identity provider
for an account to recover the secret encryption key (for instance, if
compelled to do so by legal authorities) and thus decrypt the shielded
balance and transfers on that account.)
An account can transfer an amount from its shielded balance to another
account. While the sender and receiver are publicly visible, the exact
amount transferred (in CCD) is not. An account can also decrypt funds from
its shielded balance, enabling them to be spent as normal. Each of
these operations makes use of zero-knowledge proofs to ensure that
the transfers are carried out correctly (i.e. without creating or
destroying any CCDs) while not revealing the encrypted values.
This feature has not seen widespread use. Moreover, the presence of
shielded transfers is seen as a barrier to adoption of Concordium
by some risk-averse parties. As such, the shielded transfers are
no longer supported as of protocol version 7.
In particular, shielding (or encrypting) a public balance and
transferring shielded funds from one account to another are
disallowed from protocol version 7. Unshielding (i.e., transferring
funds from an account's shielded funds to its own unshielded funds)
remains enabled. This ensures that accounts do not lose any funds that
they previously shielded.
* * *
A block hash is a cryptographic hash of the contents of a block, and serves
as a unique identifier for the block. The block hash is not only derived from
the header fields of a block, but also the state of the chain after executing
the block, including the current state of each account and smart contract.
The block hashing scheme determines how the block hash is derived from this
data. In particular, individual elements are hashed and their hashes are
combined and hashed again into a Merkle tree structure. This structure makes
it possible to prove that a block (whose hash is known to the verifier)
contains some specific data (such as the balance of a particular
account) without providing the entirety of the data comprising the block.
In protocol version 7, the block hashing scheme is redefined. The revised
definition makes certain Merkle proofs shorter and simpler. The purpose of
this is to support "light clients": light-weight verifiers that can check
properties of the chain without running the full consensus, by making use of
Merkle proofs.
Changes
The following behaviors are changed in protocol version 7.
1. The cost metering for smart contract execution is redefined. The new
instruction costs of WebAssembly instructions are defined below
(1 Energy = 1000 Interpreter Energy):
Instruction Old cost (Interpreter Energy) New cost (Interpreter Energy)
nop 1 1
unreachable 0 0
block 0 0
loop 0 0
if 10 4
br 8 + target arity (0 or 1) 2
br_if 10 4
br_table 10 + target arity 7
return 8 + return arity 2
call 26 + #arg + #res 6 + #arg + #res
call_indirect 28 + 2*(#arg + #res) 8 + floor(1.1*(#arg + #res))
drop 2 0
select 3 2
local.get 3 0
local.set 3 0
local.tee 3 0
global.get 3 1
global.set 3 1
inn.load* 4 1
i32.store 8 2
i64.store 10 2
i32.store8 5 2
i32.store16 8 2
i64.store8 7 2
i64.store16 9 2
i64.store32 10 2
memory.size 4 1
memory.grow 10 + 100 * #pages 10 + 100 * #pages
inn.const 2 0
inn.eqz 3 1
inn.clz 3 1
inn.ctz 3 1
inn.popcnt 3 1
inn.eq 4 1
inn.ne 4 1
inn.lt* 4 1
inn.gt* 4 1
inn.le* 4 1
inn.ge* 4 1
inn.add 4 1
inn.sub 4 1
inn.mul 5 2
inn.div* 5 2
inn.rem* 5 2
inn.and 4 1
inn.or 4 1
inn.xor 4 1
inn.shl 4 1
inn.shr* 4 1
inn.rotl 4 1
inn.rotr 4 1
i32.wrap_i64 3 1
inn.extend* 3 1
(Here, target arity and return arity are the arity of the target
label and the current activation frame respectively; #arg and
#res are the numbers of arguments and results of the function
being called; and #pages is the number of memory pages requested
by memory.grow. "inn" indicates both "i32" and "i64" variants, and
where instructions are suffixed with *, the line applies to all variants.
For instance, inn.load* includes i32.load8_s, i64.load8_u, etc.)
The cost (in Energy) of looking up a contract module is reduced from
floor(size / 50) to floor(size / 500), where size is the size of the
module's WebAssembly source in bytes, not including custom sections.
This cost is incurred on contract initialization, update, upgrading and
inter-contract calls.
2. The maximum nesting depth of V1 contract calls is limited to 384.
3. In V1 smart contracts, the `invoke` host function allows for two
additional operations:
- Query smart contract module reference:
* Instruction tag: 7
* Payload:
- contract index (8 bytes, little endian)
- contract subindex (8 bytes, little endian)
* Return codes:
- 0: success
the return value is the module reference of the specified
contract instance
- 3: missing contract
there is no contract instance with the given address
* Cost: 200 Energy
- Query smart contract name:
* Instruction tag: 8
* Payload:
- contract index (8 bytes, little endian)
- contract subindex (8 bytes, little endian)
* Return codes:
- 0: success
the return value is the name of the `init` method used to
create the contract instance (including the "init_" prefix)
- 3: missing contract
there is no contract instance with the given address
* Cost: 200 Energy
4. The following changes apply to staking:
- The stake of an account is divided into
* Active stake: this determines the validator or delegator's
stake for future snapshots that determine the stake
distribution for the purposes of lottery calculation and
reward distribution.
* Inactive stake: this does not contribute to lottery or
reward calculations, but is locked for a period of time.
The inactive stake can consist of multiple amounts that
can be in the following states:
- cooldown with a specific expiry time
- pre-cooldown
- pre-pre-cooldown
- When a validator or delegator is removed from an account, the
validator or delegator record is removed from the account and
the active stake is moved to the inactive stake in the
pre-pre-cooldown stake.
- When a validator or delegator has its stake reduced, the amount
of the reduction is moved to the inactive stake in the
pre-pre-cooldown stake.
- When a validator or delegator has its stake increased (including if
an account becomes a validator or delegator when it was neither),
the amount of the increase will be taken preferentially from
inactive stake in the following order:
* pre-pre-cooldown
* pre-cooldown
* cooldown (starting from the highest expiry timestamp)
- A validator or delegator may switch to the other role at any time
(by a `ConfigureBaker` or `ConfigureDelegator` transaction). It is
only subject to cooldown in this process insofar as the stake is
reduced.
- At the first block in an epoch, the following occur:
* If the epoch is the first of a new payday (a payday epoch) then
all amounts in cooldown that are set to expire at or before the
transition time for the previous epoch (the payday time) are
released from the inactive stake of accounts. All amounts in
pre-cooldown are moved into cooldown, with the expiry time set
to be the minimum of the current validator and delegator cooldown
time parameters (accounting for any parameter changes up to the
current block) after the payday time.
[Note: as before, at payday the current epoch stakes and capital
distribution are updated based on the snapshot taken in the
previous epoch, and all rewards are paid out that have accrued.]
* If the epoch is the last before a new payday (a snapshot epoch)
then all amounts in pre-pre-cooldown are transferred to pre-cooldown.
[Note: as before, in a snapshot epoch, the snapshot of the stakes
and capital distribution is taken for the next payday.]
5. The transaction payload types `TransferToEncrypted`,
`EncryptedAmountTransfer` and `EncryptedAmountTransferWithMemo` are
considered invalid. Transactions of these types will be rejected with
a `SerializationFailure` result.
6. The block hashing for a (non-genesis) block in protocol version 7 follows
the structure below:
[BlockHash]
|- protocol version
|- [...]
|- [BlockHeaderHash]
| |- round
| |- epoch
| |- [parent BlockHash]
|
|- [BlockQuasiHash]
|- [metaHash]
| |- [bakerInfoHash]
| | |- [timestampBakerHash]
| | | |- timestamp
| | | |- baker ID
| | |
| | |- [nonceHash]
| | |- block nonce
| |
| |- [certificatesHash]
| |- [QuorumCertificate hash]
| | |- [parent BlockHash]
| | |- QC round
| | |- QC epoch
| | |- aggregate signature
| | |- signatories
| |
| |- [timeoutFinalizationHash]
| |- [Option TimeoutCertificate hash]
| |- [Option FinalizationEntry hash]
|
|- [dataHash]
|- [transactions hash]
|- [BlockResultHash]
|- [...]
| |- [BlockStateHash]
| |- [TransactionOutcomesHash]
|
|- [...]
|- [block height info hash]
| |- absolute block height
| |- genesis index
| |- relative block height
|
|- [...]
|- [current FinalizationCommitteeHash]
|- [next FinalizationCommitteeHash]
Effects
1. At the protocol update, the following changes apply to validator and
delegator accounts:
* If the account was pending cooldown, it is removed from
the pending state and the amount of the reduction in stake will be moved
from its active stake to the pre-pre-cooldown inactive stake.
* If the account was pending removal (of the validator or delegator),
the validator or delegator record is removed from the account and the
active stake is moved to the pre-pre-cooldown inactive stake.
2. Behavior of existing V1 contract instances may be affected in the
following scenarios:
- The cost of smart contract execution is changed as described above.
- If the contract execution exceeds a nesting depth of 384 contract
calls, the execution will fail.
- If the contract uses the `invoke` operation with tags 7 or 8, prior
to the update a runtime error would be triggered; after the update
the behavior will be as defined above (querying the module reference
or name).
Protocol update instruction
The protocol update instruction is identified by the SHA256 hash of
this file. There is no auxiliary data for the update instruction.
Commentary¶
Protocol update commentary
Protocol Version: 7
Builds on Protocol Version: 6
Protocol version 7 introduces protocol changes in the following key areas:
- smart contract costs
- new smart contract queries
- modified stake cooldown behavior
- removal of shielded transactions
- redefined block hashing
This document provides commentary on each of these changes, elaborating on
the rationale for each.
1. Smart Contract Costs
The cost of a smart contract transaction (such as a contract init or update)
is a function of the WebAssembly instructions that are executed in computing
the outcome of the transaction. Each instruction is associated with a cost
(measured in "interpreter energy"). When a smart contract module is deployed
on the Concordium blockchain, the nodes apply a transformation on the
WebAssembly code that instruments it with metering instructions, which track
the interpreter energy usage during program execution. A smart contract
transaction (as with other transactions) has an energy limit associated with
it. If the energy usage during execution exceeds this limit, the transaction
will simply fail. The energy used by the transaction is ultimately paid for
by the sender of the transaction in a proportional amount of CCD, and these
fees are used to recompense validators.
The purpose of all of this is to limit the demands that individual
transactions can place on the resources of nodes in the Concordium blockchain,
specifically, execution time. Accordingly, the costs associated with
instructions are intended to be a close proxy for the actual execution time.
The costs are assigned based on benchmarking so that 1,000,000 energy
corresponds to roughly 1 second of execution time. (Note that the actual
execution time cannot be used to determine the cost of smart contract
transactions, since it will vary from node to node and consensus requires
that nodes agree on all details.)
Concordium node version 7 improves smart contract execution by employing a
compilation phase that transforms the stack-machine based WebAssembly into
a register-based intermediate representation. This representation can be
interpreted more efficiently, as operations are performed directly on their
operands, eliminating stack manipulation. The result of this is that
execution times are typically around 3 times faster. Accordingly the cost
of WebAssembly instructions is redefined to reflect this.
When a smart contract makes a call to another smart contract, the calling
contract's state remains in memory, while the callee's state is also
loaded. In the case of a long chain of such calls, this can use a lot of
memory, potentially exhausting the resources of nodes. Previously, the
energy limits were deemed sufficient to limit the resource usage. However,
in light of reduced energy costs, it was decided to impose a limit on the
depth of nested inter-contract calls to 384, to guard against resource
exhaustion. Since each live execution frame can be up to 32 MB in size,
this limits the total size of live execution frames to 12 GB. This limit
was chosen given the minimum RAM requirement for a node of 16 GB.
Further reading:
In the Concordium node implementation, the metering transformation can be
found at https://github.com/Concordium/concordium-base/blob/main/smart-contracts/wasm-transform/src/metering_transformation.rs
2. New Smart Contract Queries
Protocol version 7 introduces new query operations that are available to
smart contracts. These operations get the module reference and smart
contract name associated with a smart contract instance. This can be used
to check if a smart contract instance is an instance of a particular known
smart contract, for example, or to check if a smart contract has been
upgraded (in which case, its module reference will change). These kind of
checks can be useful when designing smart contracts to interact in a
partial-trust situation: a smart contract can authenticate the other
contracts it interacts with before determining whether to trust them.
A common question from smart contract developers (particularly those
coming from Ethereum) is how to implement a factory pattern on Concordium.
The architecture of Concordium's contracts (separating module deployment
from contract initialization) means that typically a factory pattern is
not required. However, these new contract inspection operations provide
a way of emulating the factory pattern in the cases where it is needed.
For details, see:
https://developer.concordium.software/en/mainnet/smart-contracts/guides/factory-pattern.html
3. Stake Cooldown Changes
The changes to stake cooldown have two main effects. First, when validators
and delegators reduce (or remove) stake, the amount of the reduction no
longer contributes to the calculations that determine lottery power and
rewards (after the following payday), but is held in a cooldown state
where it cannot be spent. Second, while stake is in cooldown, further
changes to the stake can be made, such as increasing it, decreasing it
further, removing it, or even changing between validator and delegator.
The first change brings Concordium closer in line with common industry
practice, while also strengthening a buffer against nothing-at-stake
attacks, since a malicious validator cannot sell their stake for a
certain time after they no longer have any effective power in the
proof-of-stake consensus.
The second change gives validators and delegators much more flexibility.
Previously, while stake was in cooldown, the stake could not be changed
(neither up nor down) and it was impossible to transition directly between
being a validator or a delegator. In protocol version 7, both of these
are possible. This change is of particular benefit to custodial staking
providers, who may hold stake for multiple customers on a single (validator)
account. The new arrangement allows them to withdraw funds for different
customers independently. For instance, under the old scheme, if the staking
provider unstaked customer A's funds, and then wished to also unstake
customer B's funds, then they would have to wait two full cooldown periods
before B's funds were unstaked, no matter how soon they requested them after
A's funds were already being unstaked. In the new scheme, the fact that
A's funds are in cooldown does nothing to prevent B's funds also moving to
cooldown.
4. Removal of Shielded Transactions
Shielded transactions allow accounts to send funds to each other while
obscuring the exact amount of the transfer. Part of the rationale for
supporting such transfers was to enhance the privacy for users, since the
blockchain is an entirely public record.
This feature has not seen wide adoption (at time of writing, less than
200000 CCD is shielded) and has generally been more confusing than
beneficial for users, who may have mistakenly shielded their balances
unnecessarily, without properly understanding the use of shielded transfers.
Furthermore, financial institutions with obligations under
anti-money-laundering laws may be hesitant to adopt Concordium under the
impression that such a feature could possibly enable money laundering.
In fact, the identity disclosure authorities and identity providers can,
if necessary, unmask the shielded transfers on an account, providing a
deterrent against and remedy for any use of shielded transfers for money
laundering.
For these reasons, it was decided to remove shielded transfers, disabling
the ability to shield balances and to transfer shielded amounts. However, as
shielded balances are encrypted, there is no way to simply unshield balances
automatically. Therefore, the ability to unshield balances remains, and
existing shielded balances on accounts will remain.
5. Redefined Block Hashing
The revision of the block hashing scheme in protocol version 7 is designed
to simplify Merkle proofs of various properties of blocks, such as:
- The outcome of a particular transaction in a block.
- Which finalization committee certifies blocks in the next epoch.
- That the (partial) state of a specific smart contract has a particular
value.
These proofs support and provide utility for light clients. A light client
knows the current block height and the finalization committees for the current
and next epochs. The client can update its state given a finalization proof
(signed by one of the finalization committees) for a block, and a Merkle
proof that establishes the new block height and finalization committees.
Implementing light clients as smart contracts allows trustless bridging
between suitable blockchains. Such a bridge uses light clients to validate
messages sent between chains, eliminating the need to trust third-party
relayers. Instead, the consensus protocol itself guarantees the integrity
of the messages. A Concordium light client would verify cryptographically
that the finalization committee saw the message being sent from the Concordium
chain. These changes to block hashing are thus a step towards connecting
Concordium to the wider blockchain ecosystem, for instance using the
Inter-Blockchain Communication (IBC) protocol (https://www.ibcprotocol.dev).
The same technology behind light clients can also support lightweight nodes.
Currently, Concordium nodes maintain the entire state history of the
blockchain, which is a significant and ever-growing quantity of data.
(As of writing, this amounts to approximately 115 GB of data for mainnet.)
Moreover, starting a new node from scratch is extremely time-consuming,
since the node executes every one of the millions of blocks in sequence.
A lightweight node would only store the recent state and an abbreviated
summary of historical blocks, sufficient to prove the correctness of the
current state. This would allow for faster node catch-up and reduced
storage requirements.