1) The message protocol document (rev 5.0) refers to set the network key using message id 0x46 (with only 8 bytes of the key) and doesn't make any reference for the id 0x76 as you said before. Is that a hidden message id? Where can I find more info about that?
The ANT chips typically found inside mobile phones support 128 bit/16 byte network keys (0x76 message), while all other ANT chips generally only support 64 bit/8 byte keys (0x46 message). It isn't hidden, but only applicable to Android, so you will find details in the "Creating ANT Android Applications" document in the SDK. Note that the new "channel" API does not yet support private networks, so there are no details on this in the current version of the document.
2) Is it possible to set only one channel to a private network while the other channels still using the public ANT+ network key? (I am asking that because I've noticed that SRM Power Control 7 can be paired with Sunnto belt and at the same time with other sensors using ANT+ public network).
Yes, each channel is individually allocated to the network it requests. Most ANT chips support 3 concurrent networks, while some newer parts (like the USB-m stick) support 8.
Again, note that the new "channel" API does not yet support private networks.
3) I have tried to simulate a private network with AntWare II, but I can only set 8 bytes of the network key, so I couldn't find any difference. Does ANT+ provide any software to simlate that?
There is a 8 byte and 16 byte network key for each network. There are no ANT chips you can connect to ANTware II which use a 16 byte key, so you will always use the 8 byte key in that case. If you use the matching 16 byte key on an Android device, it would connect to the same network.
4) How do I know the AOV firmware version of my Xperia Live Walkman and Xperia Arc? Is it possible to know using your API?
This is provided through the ANT Version requested response message. Note that with the new "channel" API you do not need to check specific version strings, as this is provided through the channel Capabilities.