- This article was first published in LinkedIn on November 18, 2017 https://www.linkedin.com/pulse/high-fees-hell-bitcoin-mining-explained-roberto-capodieci/
We all know Bitcoin. We all know that proof of work (aka “mining”) is necessary for Bitcoin to exist. We all know that mining burns much electricity and that mainly is done in China.
How does mining work? What is the mining difficulty everyone is talking about? Why transaction fees go so high? What is a mempool? Why mining, as it is done now, centralise Bitcoin? Where did Satoshi Nakamoto fail in designing a decentralised system?
What does “mining” mean, in Bitcoin terms?
We all know that a blockchain is a “series of blocks of data, connected (chained) one to the other thanks to cryptography”. We all know that a cryptographic hash is a unique string/number produced by a hashing algorithm applied to a source piece of data. In other words, a photo, a text file, a video… each has a unique hash. Changing the source data of a single bit changes the resulting hash. What does this mean for the blockchain?
Each block, containing a group of transactions posted in the blockchain (for example “account X sends Z bitcoin to account Y”), contains the hash of the previous block. This means that when creating a new block, a miner must include together with the transactions and his account, also the hash of the previous block. Once closing the block and finding a special hash for the block he just created, the miner is adding a new block to the (block)chain. In short, the work of the miner is the one of creating a special hash of the block he wants to close.
Why hashing uses a lot of processing power, and thus electricity?
Calculating the hash of a piece of data (a block, in the Bitcoin blockchain case), is not an excessively difficult process. Yet the proof of work protocol used by Bitcoin adds a “difficulty” to the process of calculating the hash: it requires miners to create a hash that begins with a particular string of text. As mentioned above, changing a single bit from a piece of data produces a completely different hash, thus the miner needs to do a trial and error exercise creating hashes over and over using each time a different nonce (additional piece of data) in the block they want to close, until they find a hash that matches the requested initial string of text. As the first miner to claim a block gets rewarded with some new bitcoin, miners use each possible trick to find the correct hash as soon as possible.
An example: given the block containing the text “Sample transaction from Roberto Capodieci?—?Test 0” the resulting SHA256 hash is: “070d6c61755c3caf013a176ae2cac59087acc01f1c145fc8ddf7aefb2023dabf”
(try the hash creation here: https://t.me/joinchat/CJN3VAur_epQLqlZefh2gA).
If the rule for the current difficulty is to find a hash that begins with 11, we need to keep the text “Sample transaction from Roberto Capodieci?—?Test ” unchanged, and change only our nonce, the number after “Test ”, and try sequentially until we find a hash that starts with 11
…Test 0 = 070d6c61755c3caf…
…Test 1 = b8602b011b6aeb6d…
…Test 2 = 9f24c69f3bed0946a…
…Test 3 = f546236b4d3048b6…
[49 more attempts]
…Test 53 = 11023daf11ff0454fc…
It has taken 54 attempts to find a hash that begins with 11, thus valid?—?by this example consensus rules?—?to claim the block. If the difficulty was set to find a hash that starts with “1111” it would have taken many more attempts.
The pulse of the Bitcoin network
The Bitcoin network aims to create a new block every 10 minutes. The protocol, every 2 weeks circa, adjusts the difficulty in such manner that the hashing power of the network takes 10 minutes on average to find the right hash to close a new block. This means that the difficulty is adjusted to the fastest miner, and in my opinion, this is the issue with the Bitcoin proof of work protocol.
Why making it complex to find a hash to close a block?
This is the first question someone may have before we look at the tricks used by miners to find hashes fast. Making it difficult to mine a block?—?requiring an average of 10 minutes before finding a good hash?—?is done to protect the blockchain from being rewritten, as it would make it very hard, if not impossible, to catch up with the constantly growing real blockchain.
I mined 6 blocks with a small notebook, in 2010 (and then I formatted the HDD).
Today it would be impossible. Why? I think Satoshi Nakamoto (the team of people that designed Bitcoin and its blockchain) never thought about mining pools. What is a mining pool? Well, in the example above it has taken 54 attempts to find the right hash. This is 54 loops of trials and error that have taken a certain amount of time. The difficulty is adjusted so that the amount of attempts needed to find a solution requires an average of 10 minutes. What if we had 54 computers working together? It would have taken us one single loop and one of the 54 computers would have had the right nonce to generate the hash to match the required difficulty rule. After a few consecutive wins generating blocks at much less than 10 minutes one from the other, the difficulty would have adjusted so that it would take us 10 minutes?—?with 54 computers?—?to find the right hash, leaving close to zero chances to people with a single computer at home to find a hash.
As mining is a profitable activity, miners invest big money in mining rigs that are not only faster than regular processors in doing the trial and error of the hashes, but have thousands of hashing devices working together to find the right hash to claim the bitcoin reward if they arrive first. Constantly the difficulty adjusts so that it takes an average of 10 minutes to those mining operations with thousands of miners working together to find the right hash. There are left about 5 mining operations that are competing with each other, leaving everyone else with no hope to participate in the game.
What has all this to do with the transaction fees?
Miners earn both the rewards for closing blocks, plus all the transaction fees of the transactions included in each of the blocks. As blocks in the Bitcoin blockchain have a maximum size of 1Mb (currently), the quantity of transactions that can be included in a block is limited. There is a mempool, a queue of transactions waiting to be included in the next blocks, and is up to the miner to choose which transactions from the mempool to include in the block he wants to close. As transaction fees are voluntarily set by whoever post a bitcoin transaction, it seems obvious that if someone wants his transaction to be included in one of the next blocks, he or she should offer an attractive fee for the miners to chose his or her transaction. Failing to do this may leave their transaction in the queue for up to a day.
What happens when there is something sexier to mine?
With forks such as Bitcoin Cash (BCH), the late Bitcoin Classic (BCC), the probably resurrected SegWit2x, and others to come, plus clones such as LiteCoin (LTC) and others, miners have a choice of what to mine. If something sexy shows up all of a sudden, see what happened with Bitcoin Cash after its value rise, miners move some of their hashing power away from Bitcoin and put it on another profitable blockchain. This cause a sudden drop of hashing power in the bitcoin blockchain and, until the difficulty adjusts, blocks slow down as it takes much more than 10 minutes to find the right hash for the current difficulty, causing an increased volume in the transaction queue, and consequently higher fees.
What is a solution to this problem?
Bitcoin Gold forked Bitcoin and changed hashing algorithms making all the big miners unable to mine it with their current hardware (they require a GPU?—?a video card?—?to mine Bitcoin Gold). This democratises mining for a while until a few people will find lucrative setting up GPU mining rigs, leaving regular people out of the game.
Bitcoin Cash made bigger blocks (up to 8Mb against the 1Mb of Bitcoin) to allow a higher number of transactions to be included in a single block. Many other blockchains use different approaches to secure their blocks creation, moving away from the proof of work used by Bitcoin, and many are excellent solutions, yet they are on other blockchain and not on Bitcoin.
So far nobody has found and deployed a solution that stops the centralization of mining.
We are thinking about a solution
At Blockchain Zoo we started brainstorming on solutions that bring back mining to every regular user. Once mining is redistributed among all the participants in the peer to peer network of a blockchain, there is no need of incredible hashing power, and even a proof of work node can run on a mobile phone. I will go more in details on the solution we are working on in another post, but if you want to know more about Blockchain Zoo and join the conversation, join our Telegram group: https://t.me/bczoo
[NOTE?—?August 2019]: the research we have put in place, for long time, lead to the creation of a project that will be presented in October 2019: ZooBC