How to create transactions that carry messages in OP_RETURN

masterOfDisaster 2015-11-09 08:49:59 UTC #1

I was looking for a method to send messages along with transactions.
I’m aware that the OP_RETURN can carry only a limited amount of data (is it 40 Bytes at Nu?), but I would only use a few bytes anyway for what I have in mind.

It seems it could be possible with raw transactions (signrawtransaction <hex string> [{"txid":txid,"vout":n,"scriptPubKey":hex},...] [<privatekey1>,...] [sighashtype="ALL"]), but I have no clue how to do that and where to start.

Any help would be appreciated.
@hyena, I hope you can point me in the right direction, because your service http://www.cryptograffiti.info/ does that for Bitcoin transactions, right?

Hyena 2015-11-09 12:16:58 UTC #2

Yes. I didn’t know OP_RETURN only allows 40 bytes. It’s weird and stupid. There should be no limit. According to http://blog.coinprism.com/2015/02/11/80-bytes-op-return/ the limit is 80 but it’s still not enough.

Anyway, in cryptograffiti the messages are converted into an array of bitcoin receiving addresses. Then, a payment is made in a single transaction to all of those addresses. Each address allows stroring 20 bytes of data. With the help of RPCs it is possible to include duplicate outputs in the same TX. By providing your own change address in the end of the TX you don’t mess up the data (otherwise bitcoin-qt puts the change address randomly in the middle of the TX).

I haven’t played around with OP_RETURN myself other than recognizing it in TXs that I attempt to decode. For example, my new WIP version of cryptograffiti decodes some OP_RETURN messages starting from message #3861 ( http://cryptograffiti.info/interface/reloaded/index.html ).

Hyena 2015-11-09 12:53:33 UTC #3

Signing the raw transaction is irrelevant at this point. You will have to construct that raw transaction at first.

createrawtransaction [{"txid":txid,"vout":n},...] {address:amount,...}

In bitcoin, OP_RETURN opcode is represented by the hex value 0x6a. One way is to search for that byte starting from the end of the raw transaction. You would append your data behind OP_RETURN and then sign the raw transaction (signrawtransaction). After having it signed you would have to broadcast the TX.

masterOfDisaster 2015-11-09 13:05:39 UTC #4

Without a limit you could bloat blockchains even faster.

So this conversion is a two-way function? Otherwise it wouldn’t be possible to derive the message from the addresses, right?

Oops, I tried to start at the wrong end!
I try to get behind what you mean, although I have no clue how to create a valid raw transaction.

I remember @Nagalim dealt with raw tx when he worked on the seeded auctions software. Maybe he can enlighten me and save me from reinventing the wheel!
Than you very much!

Nagalim 2015-11-09 13:09:50 UTC #6

3 rpc commands: Create, Sign, Send Raw Transaction. Use Create first, not Sign, like you posted in the OP.

Also be wary or you’ll mess up your wallet. Be super conscious of outputs, change addresses, and the required fee.

Hyena 2015-11-09 13:36:44 UTC #7

False. TX fee gets bigger as the size of the TX grows. Miner would simply not confirm a TX that does not pay a high enough fee.

Correct.

Here’s the Lua function I use to construct a raw transaction:

function bitcoin_fun_create_raw_transaction(inputs, outputs)
    -- Because Lua dictionaries cannot have a predefined order we must
    -- construct this request manually.
    local outputlist = "{";
    local k, v;
    
    local mapping = {};
    local order   = {};
    local index;
    local index_hex;
    local hex;
    local salt = "MPGdJA7KX9TwBB7i";
    
    for i=1, #outputs do
        k, v = next(outputs[i], nil);

        index_hex = hash_SHA256(i..salt, false);
        index_hex = string.sub(index_hex, 1, 40);
        index     = string.fromhex(index_hex);
        index     = Bitcoin.stringToAddress(index);
        hex       = Bitcoin.addressToString(k);
        hex       = string.tohex(hex);
        mapping[index_hex] = hex;
        table.insert(order, index_hex);
        
        --log("mapping["..index_hex.."] = "..hex);
        outputlist = outputlist..'"'..index..'": '..v;
        if (i < #outputs) then
            outputlist = outputlist .. ", ";
        end;
    end;
    outputlist = outputlist .. "}";
    
    local inputlist = JSON:encode(inputs);
    
    local request = '{\n'..
                    '      "id": 1,          \n'..
                    ' "jsonrpc": "2.0",      \n'..
                    '  "method": "createrawtransaction", \n'..
                    '  "params": [ '..inputlist..', '..outputlist..' ]\n'..
                    '}';
    --warn(request);
    response = url_post("http://"..bitcoin.rpc_user..":"
                                 ..bitcoin.rpc_password.."@"
                                 ..bitcoin.rpc_ip..":"
                                 ..bitcoin.rpc_port.."/", request, bitcoin.curl_timeout);
    --warn(response);
    if (response ~= nil) then
        local t = JSON:decode(response);
        if (t ~= nil and type(t) == "table") then
            if (type(t.error) == "table" and type(t.error.message) == "string" ) then
                log("Bitcoin createrawtransaction: "..t.error.message);
                notify_admin("createrawtransaction", t.error.message);
            else
                local k,v;
                local s = 1;
                local e;
                local bad = nil;
                -- Replace indexes with real output hex representations that may
                -- contain duplicates.
                for i=1, #order do
                    k = order[i];
                    v = mapping[k];

                    s, e = string.find(t.result, k, s, true);
                    if (s ~= nil and e ~= nil) then
                        t.result = t.result:sub(1, s-1) .. v .. t.result:sub(e+1);
                        s = e + 1;
                    else
                        bad = k;
                    end;
                end;
                --warn(t.result);
                
                if (bad ~= nil) then
                    local msg = "Index hex "..bad.." not found from raw transaction.";
                    warn(msg);
                    notify_admin("createrawtransaction", msg);
                    return nil;
                end;                
            end;
            return t.result;
        end;
    else
        local msg = "Bitcoin RPC failed on cURL post: createrawtransaction";
        warn(msg);
        notify_admin("createrawtransaction", msg);
    end;
    
    notify_admin("createrawtransaction", "Failed to create a raw transaction, returning nil.");
    return nil;
end

It doesn’t do anything about OP_RETURNs though and you would have to do the change address manually. If you don’t include a change address you could lose a lot of (bit)coins!

masterOfDisaster 2015-11-09 14:04:37 UTC #8

Not yet true.
As long as the tx fee is still required per kB and not per byte, you could easily bloat the blockchain much more for the same tx fee by increasing the tx size but while staying below 1 kB.

I try to understand what it does, but will likely fail not being a programmer
Thank you anyway!

Hyena 2015-11-09 14:23:26 UTC #9

The TX fee is ceiled not floored. You will always have to pay more rather than less than what is required.

The point is that the RPC natively does not allow you to create a TX that has duplicate outputs. For that reason I first create a TX that sends coins to dummy addresses. After receiving the hex I will replace the substrings with real addresses that can contain duplicates. Then I sign it and then I broadcast it. Anyway, that method is a bit offtopic since you want to utilize OP_RETURN.