Part 8: Web3.py Console¶
Geth Console in Python¶
You have probably stumbled already with the term “geth console”. Geth exposes a javascript console with the Web3 javascript API, which is handy when you need an interactive command-line access to geth and the running node. It’s also a great way to tinker with the blockchain.
Good news: you have identical interactive console with Python. All you have to do is to initiate a Web3.py connection in a Python shell, and from that point forward you have the exact same Web3 interface in Python.
We will show here only a few examples to get you going. Web3 is a comprehensive and rich API to the Ethereum platform, and you probably want to familiarise yourself with it.
Almost any Web3.js javascript API has a Pythonic counterpart in Web3.py.
See the the Web3.py documentation and the full list of `web3.js JavaScript API.
Web3.py Connection to Geth¶
Start with the horton
local chain. Run the chain:
$ chains/horton/./run_chain.sh
Start a Python shell:
$ python
Note
You may need $ python3
Initiate a Web3 object:
>>> from web3 import Web3, IPCProvider
>>> w3 = Web3(IPCProvider(ipc_path="/home/mary/projects/donations/chains/horton/chain_data/geth.ipc"))
>>> w3
>>> <web3.main.Web3 object at 0x7f29dcc7c048>
That’s it. You now have a full Web3 API access in Python.
Note
Use the actual path to the horton
ipc_path. If you are not sure, look at the run file
argument at chains/horton/run_chain.sh
.
Sidenote, this is a good example why geth uses IPC (inter-process communication). It allows other local processes to access the running node. Web3, as another process, hooks to this IPC endpoint.
Console Interaction with a Contract Instance¶
We can interact with a contract instance via the Python shell as well. We will do things in a bit convuluted manual way here, for the sake of demostration. During the regular development process, Populus does all that for you.
To get a handle to the Donator
instance on horton
we need (a) it’s address, where the bytecode sits,
and (b) the ABI ,application binary interface, the detailed description of the functions and arguments
of the contract interface. With the ABI the EVM knows how to call the compiled
bytecode.
Warning
A reminder, even when you call a contract without an ABI, or just send it Ether, you may still invoke code execution. The EVM will call the fallback function, if it exists in the contract
First, the address. In another terminal window:
$ cat registrar.json
The Registrar is where Populus holds the deployment details:
{
"deployments": {
"blockchain://927b61e39ed1e14a6e8e8b3d166044737babbadda3fa704b8ca860376fe3e90b/block/2e9002f82cc4c834369039b87916be541feb4e2ff49036cafa95a23b45ecce73": {
"Donator": "0xcffb2715ead1e0278995cdd6d1736a60ff50c6a5"
},
"blockchain://c77836f10cb9691c430638647b95701568ace603d0876ff41c6f0b61218254b4/block/34f52122cf90aa2ad90bbab34e7ff23bb8619d4abb2d8e66c52806ec9b992986": {
"Greeter": "0xc5697df77a7f35dd1eb643fc2826c79d95b0bd76"
},
"blockchain://c77836f10cb9691c430638647b95701568ace603d0876ff41c6f0b61218254b4/block/667aa2e5f0dea4087b645a9287efa181cf6dad4ed96516b63aefb7ef5c4b1dff": {
"Donator": "0xb8d9d2afbe18fd6ac43042164ece9691eb9288ed"
}
}
}
It’s hard to tell which blockchain is horton
. Populus encodes a blockchain signature
by the hash of it’s block 0. We only see that Donator
is deployed on two blockchains.
But since Greeter
was deployed only on horton
,
the blockchain with two deployments is horton
, and the other one is morty
.
So we can tell that Donator
address on horton
is
"0xb8d9d2afbe18fd6ac43042164ece9691eb9288ed"
.
Copy the actual address from your registrar file. Back in the python shell terminal, paste it:
>>> address = "0xb8d9d2afbe18fd6ac43042164ece9691eb9288ed"
Now we need the ABI. Go to the other terminal window:
$ solc --abi contract/Donator.sol
======= contracts/Donator.sol:Donator =======
Contract JSON ABI
[{"constant":true,"inputs":[],"name":"donationsCount","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"donationsUsd","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"defaultUsdRate","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"donationsTotal","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"usd_rate","type":"uint256"}],"name":"donate","outputs":[],"payable":true,"type":"function"},{"inputs":[],"payable":false,"type":"constructor"},{"payable":true,"type":"fallback"}]
Copy only the long list [{"constant":true,"inputs":[]....]
, get back to the python shell,
and paste the abi inside single quotes, like '[...]'
as follows:
>>> abi_js = '[{"constant":true,"inputs":[],"name":"donationsCount","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"donationsUsd","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"defaultUsdRate","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"donationsTotal","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"usd_rate","type":"uint256"}],"name":"donate","outputs":[],"payable":true,"type":"function"},{"inputs":[],"payable":false,"type":"constructor"},{"payable":true,"type":"fallback"}]'
Your python shell should look like this:
>>> from web3 import Web3, IPCProvider
>>> w3 = Web3(IPCProvider(ipc_path="/home/mary/projects/donations/chains/horton/chain_data/geth.ipc"))
>>> w3
>>> <web3.main.Web3 object at 0x7f29dcc7c048>
>>> address = "0xb8d9d2afbe18fd6ac43042164ece9691eb9288ed"
>>> abi_js = '[{"constant":true,"inputs":[],"name":"donationsCount","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"donationsUsd","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"defaultUsdRate","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"donationsTotal","outputs":[{"name":"","type":"uint256"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"usd_rate","type":"uint256"}],"name":"donate","outputs":[],"payable":true,"type":"function"},{"inputs":[],"payable":false,"type":"constructor"},{"payable":true,"type":"fallback"}]'
From now on, we will stay in the python shell.
Solc produced the ABI in JSON. Convert it to Python:
>>> import json
>>> abi = json.loads(abi_js)
Ready to instanciate a contract object:
>>> donator = w3.eth.contract(address=address,abi=abi)
>>> donator
<web3.contract.Contract object at 0x7f3b285245f8>
You now have the familiar donator
Python object, with Python methods, that corresponds to a deployed contract
instance bytecode on a blockchain.
Let’s verify it:
>>> donator.call().donationsCount()
4
>>> donator.call().donationsTotal()
8000000000000000000
Works.
Btw, everything you did so far you can do in Populus with one line of code:
donator, deploy_tx_hash = chain.provider.get_or_deploy_contract('Donator')
When you work with Populus, you don’t have to mess with the ABI.
Populus saves the ABI with other important compilation info at build/contracts.json
,
and grabs it when required, gets the Web3 handle to the chain, creates the contract
object and returns it to you (and if the contract is not deployed, it will deploy it).
Warning
Worth to remind again. Never call a contract with just an address and an ABI. You never know what the code at that address does behind the ABI. The only safe way is either if you absolutly know and trust the author, or to check it yourself. Get the source code from the author, make sure the source is safe, then compile it yourself, and verify that the compiled bytecode on your side is exactly the same as the bytecode at the said address on the blockchain.
Note that donationsTotal
is not the account balance as it is saved in the blockchain state.
It’s rather a contract state. If we made a calculation mistake with donationsTotal
then it won’t
reflect the actual Ether balance of the contract.
So all these doantions are not in the balance? Let’s see:
>>> w3.eth.getBalacne(donator.address)
8000000000000000000
Phew. It’s OK. The donationsTotal
is exactly the same as the “official” balance.
And in Ether:
>>> w3.fromWei(w3.eth.getBalance(donator.address),'ether')
Decimal('8')
How much is left in your coinbase account on horton
?
>>> w3.fromWei(w3.eth.getBalance(w3.eth.coinbase),'ether')
Decimal('1000000026682')
Oh. This is one of those accounts where it’s fun to check the balance, isn’t it?
So much Ether! why not donate some? Prepare a transaction:
>>> transaction = {'value':5*(10**18),'from':w3.eth.coinbase}
Only 5 Ether. Maybe next time. Reminder: The default unit is always Wei, and 1 Ether == 10 ** 18 Wei.
Send the transaction, assume the effective ETH/USD exchange rate is $7 per Ether:
>>> donator.transact(transaction).donate(7)
'0x86826ad2df93ffc6d6a6ac94dc112a66be2fff0453c7945f26bcaf20915058f9'
The hash is the transaction’s hash. On local chains transactions are picked and mined in seconds, so we can expect to see the changed state almost immidiately:
>>> donator.call().donationsTotal()
13000000000000000000
>>> donator.call().defaultUsdRate()
7
>>> w3.fromWei(w3.eth.getBalance(donator.address),'ether')
Decimal('13')
You can also send a transaction directly to the chain, instead of via the donator
contract object.
It’s a good opportunity, too, for a little more generousity. Maybe you go through the roof and donate
100 Ether!
>>> transaction = {'value':100*(10**18),'from':w3.eth.coinbase,'to':donator.address}
>>> w3.eth.sendTransaction(transaction)
>>> '0x395f5fdda0be89c803ba836e57a81920b41c39689ffefaaaaf6a30f532901bf5'
Check the state:
>>> donator.call().donationsTotal()
113000000000000000000
>>> donator.call().defaultUsdRate()
7
>>> w3.fromWei(w3.eth.getBalance(donator.address),'ether')
Decimal('113')
Now pause for a moment. What just happened here? The transaction you just sent didn’t call the donate
function
at all. How did the donations total and balance increased? Take a look at the transaction again:
>>> transaction = {'value':100*(10**18),'from':w3.eth.coinbase,'to':donator.address}
No mention of the donate
function, yet the 100 Ether were transfered and donated. How?
If you answered fallback you would be correct. The contract has a fallback function:
// fallback function
function () payable {
donate(defaultUsdRate);
}
A fallback function is the one un-named function you can optionally include in a contract. If it
exists, the EVM will call it when you just send Ether, without a function call. In Donator
,
the fallback just calls donate
with the current defaultUsdRate
. This is why the balance
did increase by 100 Ether, but the ETH/USD rate didn’t change
(there are also other options to invoke the fallback).
Unlike a transaction, call
doesn’t change state:
>>> transaction = {'value':50,'from':w3.eth.coinbase}
>>>> donator.call(transaction).donate(10)
[]
>>> w3.fromWei(w3.eth.getBalance(donator.address),'ether')
>>> Decimal('113')
>>> donator.call().defaultUsdRate()
>>> 7
Console Interaction With Accounts¶
List of accounts:
>>> w3.eth.accounts
['0x66c91389b47bcc0bc6206ef345b889db05ca6ef2']
Go to another terminal, in the shell:
$ ls chains/horton/chain_data/keystore
UTC--2017-10-19T14-43-31.487534744Z--66c91389b47bcc0bc6206ef345b889db05ca6ef2
The account hash in the file name should be the same as the one you have on the Python shell. Web3 got the account from geth, and geth saves the accounts as wallet files in the ‘’keystore`` directory.
Note
The first part of the wallet file is a timestamp. See Wallets
Create a new account:
>>> w3.personal.newAccount()
Warning: Password input may be echoed.
Passphrase:demopassword
Warning: Password input may be echoed.
Repeat passphrase:demopassword
'0x7ddb35e66679cb9bdf5380bfa4a7f87684c418d0'
A new account was created. In another terminal, in a shell:
$ ls chains/horton/chain_data/keystore
UTC--2017-10-19T14-43-31.487534744Z--66c91389b47bcc0bc6206ef345b889db05ca6ef2
UTC--2017-10-21T14-08-01.257964745Z--7ddb35e66679cb9bdf5380bfa4a7f87684c418d0
The new wallet file was added to the chain keystore.
In the python shell:
>>> w3.eth.accounts
['0x66c91389b47bcc0bc6206ef345b889db05ca6ef2', '0x7ddb35e66679cb9bdf5380bfa4a7f87684c418d0']
Unlock this new account:
>>> w3.personal.unlockAccount(account="0x7ddb35e66679cb9bdf5380bfa4a7f87684c418d0",passphrase="demopassword")
True
Warning
Tinkering with accounts freely is great for development and testing. Not with real Ether. You should be extremely careful when you unlock an account with real Ether. Create new accounts with geth directly, so passwords don’t appear in history. Use strong passwords, and the correct permissions. See A Word of Caution
Getting Info from the Blockchain¶
The Web3 API has many usefull calls to query and get info from the blockchain. All this information is publicly available, and there are many websites that present it with a GUI, like etherscan.io. The same info is available programmaticaly with Web3.
Quit the horton
chain and start a new Python shell.
Mainnet with Infura.io¶
As an endpoint we will use infura.io
. It’s a publicly avaialble blockchain node, by Consensys, which is great
for read-only queries.
Infura is a remote node, so you will use the HTTPProvider
.
Note
Reminder: IPC, by design, allows only local processes to hook to the endpoint. Proccesses that run on the same machine. IPC is safer if you have to unlock an account, but for read-only queries remote HTTP is perfectly OK. Did we asked you already to look at A Word of Caution? We thought so.
Start a Web3 connection.
>>> from web3 import Web3,HTTPProvider
>>> mainent = Web3(HTTPProvider("https://mainnet.infura.io"))
>>> mainent
<web3.main.Web3 object at 0x7f7d3fb71fd0>
Nice. You have an access to the entire blockchain info at yor fingertips.
Get the last block:
>>> mainent.eth.blockNumber
4402735
Get the block itself:
>>> mainent.eth.getBlock(4402735)
AttributeDict({'mixHash': '0x83a49ac6843 ... })
The number of transactions that were included in this block:
>>> mainent.eth.getBlockTransactionCount(4402735)
58
The hash of the first transaction in this block:
>>> mainent.eth.getBlock(4402735)['transactions'][0]
'0x03d0012ed82a6f9beff945c9189908f732c2c01a71cef5c453a1c22da7f884e4'
This transaction details:
>>> mainent.eth.getTransaction('0x03d0012ed82a6f9beff945c9189908f732c2c01a71cef5c453a1c22da7f884e4')
AttributeDict({'transactionIndex': 0, 'to': '0x34f9f3a0e64ba ... })
Note
If you will use block number 4402735, you should get exactly the same output as shown above.
This is the mainent
, which is synced accross all the nodes, and every node will return the same info.
The local chains horton
or morty
run a private instance, so every machine produces it’s own blocks and hashes.
Not so on the global, real blockchain, where all the nodes are synced
(which is the crux of the whole blockchain idea).
Testnet with Infura.io¶
Web3 exposes the API to the testnet, just by using a different url:
>>> from web3 import Web3,HTTPProvider
>>> testnet = Web3(HTTPProvider("https://ropsten.infura.io"))
>>> testnet
<web3.main.Web3 object at 0x7ff597407d68>
>>> testnet.eth.blockNumber
1916242
Mainnet and Testnet with a Local Node¶
In order to use Web3.py to access the mainnet
simply run geth on your machine:
$ geth
When geth starts, it provides a lot of info. Look for the line IPC endpoint opened:
, and use this IPC endpoint path
for the Web3 IPCProvider.
In a similar way, when you run:
$ geth --testnet
You will see another ipc path, and hooking to it will open a Web3 instance to the testnet
.
Note
When you run geth for the first time, syncing can take time. The best way is to just to let geth run without interuption until it synced.
Note
Geth keeps the accounts in the keystore
directory for each chain. If you want to use the same account,
you will have to copy of import the wallet file. See Managing Accounts in geth
Interim Summary¶
- Interactive Web3 API in a Python shell
- Interacting with a contract instance in the Python shell
- Managing accounts in the Python shell
- Query the blockchain info on
mainnet
andtestnet
with HTTP from a remote node - Query the blockchain info on local geth nodes.
Although Populus does a lot of work behind the scenes, it’s recommended to have a good grasp of Web3. See the the Web3.py documentation and the full list of `web3.js JavaScript API. Most of the javascript API have a Python equivalent.