Friday 26 February 2016

Informative post by Greg Maxwell on new blocksonly feature, limits of efficient block transfer and better relay methods

Couple of highlights

  • Bitcoin Core 0.12 introduced a new blocksonly setting. When set to blocksonly a node behaves normally but sends and receives no lose transactions; instead it handles only complete blocks. There are many applications for nodes where only confirmed transactions are interesting, and a node which still verifies and forwards blocks still contributes to network health-- less, perhaps, than one that relays transactions: but it also consumes fewer resources to begin with. An additional downside they don't get the latency advantages of signature caching since every transaction they see is totally new to them-- this isn't something miners should use.

  • How much less bandwidth does blocksonly use in practice? I recently measured this using two techniques: Once by instrumenting a node to measure bandwidth used for blocks vs all other traffic, and again by repeatedly running in both modes for a day and monitoring the hosts total network usage; both modes gave effectively the same result.

    • How much is the savings? Blocksonly reduced the node's bandwidth usage by 88%.
  • A significant implication of this is that any scheme for bandwidth reduction which works by using already relayed transactions to reduce the size of transmitted blocks can AT MOST reduce the overall bandwidth usage by 12%-- assuming that differential compression achieved an "infinity-fold" improvement.

  • Why does relay use so much bandwidth? The fundamental reason for this is because relay in the Bitcoin Protocol is linear in the number of transactions and the number of peers. (E.g. bandwidth is a function of Transactions * Peers).

One possible scheme I've been discussing (and working on) for a while is mempool reconciliation.

  • The idea is this, at random intervals a node will ask one of it's peers for a sketch of the top X MBytes of it's mempool. In it's request it could also signal that it's interested only in transactions whos ancestor feerate is over some minimum threshold, along with information about the size and feerate of its own mempool. The returned sketch is an IBLT of some size estimated by the sender based on the information sent by the requester.

  • Unlike the efficient block transmission IBLT proposals, this IBLT only transmits transaction IDs-- which will avoid the fragmentation overhead needed in sending whole transactions.

  • The requester then attempts to reconstruct the IDs using the content of it's own mempool. If it is unable to reconstruct all of the IDs, it requests from another random peer with a different seed.

  • When the node has multiple IBLTs it can use the partial solutions from each of them to help solve the other ones.

  • As it learns new txids that it was previously unaware of it then fetches them via getdata. When it completes a reconstruction it can INV any missing transactions towards the peers that didn't have them.

In any case, this scheme would avoid the quadratic-like behavior of relay bandwidth. It would let nodes trade latency off vs relay overhead, and it would allow for a graceful way of handling the highest priority transactions first. -- transactions a dozen or more blocks deep in the mempool are not going to get mined any time soon, so if they have lower relay latency from reconciling them less frequently that is no great harm. I believe it could do so without substantial increases in vulnerability to attack. The same software infrastructure could also be used for bandwidth minimized block transfer for nodes that do have mempools (otherwise blocksonly gives optimal bandwidth gains), though latency minimization is much better accomplished via techniques like Matt's efficient block relay protocol-- since the most important thing to minimize latency is to minimize roundtrips.

Full post http://bit.ly/1oDT8IS



Submitted February 26, 2016 at 01:02PM by brg444 http://bit.ly/24su9t1

No comments :

Post a Comment