Transparency, reproducible vote tallies etc. - digging in to the blockchain

I’m trying to learn more about the civil community and reproduce the results that we’re reading about (e.g. results of the votes) directly from the blockchain. (That’s the whole point of blockchains, right!?!)
I’ve asked for some help but it hasn’t been at the level I needed. So after some more experimentation, I’ll ask some more here. I originally posted this in slack, but this seems like a better place now.

One of my goals is to independently calculate the results of the first challenge, and reproduce the report that there were 15.00 CVL voting for “accept” and 684998.97 CVL to “reject”.
Can anyone point to data to support that on a site like etherscan?

I’ve installed the web3.py library so I have more flexibility and can automate such things via Python.
It seems to work in that I can run e.g. cvl.functions.name().call() and get ‘CivilTCR’ (identifying the underlying technology as a Token curated registry).
But I get errors when I ask other things that I would assume to work, like cvl.functions.totalSupply().call():
web3.exceptions.BadFunctionCallOutput: Could not decode contract function call totalSupply return data b’’ for output_types [‘uint256’]
Looking the error up online leads to a post (https://github.com/ethereum/web3.py/issues/518) that speculates that there’s something unusual about the contract.
Is there anyone here that can point to similar independent work to clarify how the civil blockchain can be interpreted, or help me figure out what I’m doing wrong?
Or perhaps point to the exact code (assuming it is open-source and online, e.g. somewhere at https://github.com/joincivil/Civil) which calculates results of votes?

1 Like

Here are some notes from the slack discussion at Slack | open-discussion | Neal McBurnett 2019-04-23T21:01 | Civil Community, thanks to input from Nick Reynolds, Walker and others.

Overview:

  • It’s not one contract but a network of them. There’s the tcr contract, the token contract, the voting contract, etc. We have a js library for interacting with them @joincivil/cor
  • In ENS, civiltcr.eth resolves to the tcr contract
  • The best way to get to other stuff like the token/plcr contract address is to go through that contract name
  • PLCR stands for partial lock commit reveal voting
  • voting transaction history: https://etherscan.io/txs?a=0x55656b8a58df94c1e8b5142f8da973301452ea65
  • check events on the CivilPLCRVoting to confirm vote totals
  • The challengeID on the listing will correspond to a pollID on the CivilPLCRVoting contract
  • the challenge mapping doesn’t map newsroom addresses to challenge. you can find out if a newsroom has a challenge currently by checking the listings mapping, which should return an array
  • to get historical data you consult the event logs

Relevant code:

[posting in separate messages since I am still a “new user” with restricted ability to post links…]

I don’t really have experience with web3 and python, so you are going to have to do a lot of recreating the wheel here. I’ll probably drop a bunch of links to some code and hopefully that can start pointing you in the right direction.

All of the ABI to the contracts live here:


Contracts you will likely be interested in are CVLToken, CivilPLCRVoting, and CivilTCR

Contract source code lives here:

So from there we have 2 ways of calculating the result. The first is in Typescript, and please be aware - this is a lot of code here and is pretty confusing.In short, we have the core package look at all of the ABI artifacts and generates a bunch of Typescript bindings to call the web3 functions. These are generated in packages/core/src/contractsgenerated. Almost everything happens through aggregating contract “events”. The codegen produces Observables that are then processed in subsequent classes. This is a good example:

I am sure you could make this much much simpler using just like ethers.js and passing in the ABI to the TCR contract, then just filtering on _VoteRevealed for the specified pollID.

Our Golang code is a bit more straightforward:


I don’t think this repo does any aggregation though.

1 Like

Hey Neal, feel free to share your python code in a Github gist (or in a repo) and we can take a look that way. It might be helpful to see what you are attempting to do in code.

1 Like

Still lots to do to learn and do to make this more transparent, but this is a start…

Good idea. Here you go: Civil Transparency Experiments in Python: Jupyter Notebook in a gist

Thanks Neal, on my initial glance of your code, the call to retrieve the challenge should be using the challengeID, not the address of the newsroom. You should be able to get the latest challengeID on a listing by retrieving the challengeID value from a listing.

You can see this in our contracts at: https://github.com/joincivil/Civil/blob/master/packages/contracts/contracts/tcr/AddressRegistry.sol#L35
https://github.com/joincivil/Civil/blob/master/packages/contracts/contracts/tcr/AddressRegistry.sol#L48

The code I’m seeing:

trailbl = Web3.toInt(hexstr='0x49db84c3dbea7240293c7cd943827d59a8dc766b')
cvl.functions.challenges(trailbl).call()

Should be something like (untested pseudocode):

listingAddr = Web3.toInt(hexstr='0x49db84c3dbea7240293c7cd943827d59a8dc766b')
listing = cvl.functions.listings(listingAddr)
challengeID = Web3.toInt(listing.challengeID)
cvl.functions.challenges(challengeID).call()

I can help you dig in more when I have time.

@dan @nick does this sound right?

1 Like

so yes in order to get challenge data you need to use the challengeID, and you can get that for an active challenge from the listing, but once a challenge has ended, the challengeID field is zeroed out on the listing, so it would return 0. If you want to get previous challengeIDs you’ll have to check the events emitted (and can filter on a specific newsroom address)

2 Likes

Cool, thanks Nick, that makes sense.

@nealmcb you can take a look at the documentation on filtering event logs: https://web3py.readthedocs.io/en/stable/filters.html#event-log-filters

Thanks, @petersng. I tried that pseudocode and on this line:

listing = cvl.functions.listings(listingAddr)

I got this:

Could not identify the intended function with name `listings`, positional argument(s) of type `(<class 'int'>,)` and keyword argument(s) of type `{}`.
Found 1 function(s) with the name `listings`: ['listings(address)']
Function invocation failed due to no matching argument types.

I see listings offhand in these files:

packages/artifacts/v1/AddressRegistry.json:      "name": "listings",
packages/artifacts/v1/CivilTCR.json:      "name": "listings",
packages/artifacts/v1/ContractAddressRegistry.json:      "name": "listings",
packages/artifacts/v1/RestrictedAddressRegistry.json:      "name": "listings",

@nealmcb seems like we are passing in an int, not an address type. maybe try listingAddr = Web3.toChecksumAddress('0x49db84c3dbea7240293c7cd943827d59a8dc766b')?

That gets me to the next line, where I get

AttributeError: 'listings' object has no attribute 'challengeID'

Is it listings.challengeID or listing.challengeID? I assume it should be singular.

Revised to avoid confusion

The variable is named listing. The error message seems to be calling it a listings object.
Printing out the variable itself shows that it is a python function:

>>> listing
<Function listings(address) bound to ('0x49Db84C3dBeA7240293C7Cd943827d59a8dC766B',)>

I also tried this, during a moment of yet more confusion…

listing = cvl.functions.listing(listingAddr)

yields

MismatchedABI: ("The function 'listing' was not found in this contract's abi. ",
 'Are you sure you provided the correct contract abi?')

Did you try:

// Get the listing
l = cvl.functions.listings(listingAddr)
// Get the challenge
c = cvl.functions.challenges(l.challengeID)

I would just see if this runs as this should return an invalid result since, as Nick pointed out above, l.challengeID is likely set to 0 since there is currently no challenge on the listing. You will need to look through the events.

Sorry, I would just run the code, but the python I have on my machine is on 2.7 and it doesn’t seem to be cooperating right now when trying to migrate it all to 3.7.

That gives me the same error:

AttributeError: 'listings' object has no attribute 'challengeID'

You should be able to run this in the cloud for free via https://mybinder.org or https://colab.research.google.com, where I think you can run notebooks, install arbitrary packages (via !pip install web3 or whatever.

So try cvl.functions.listings(listingAddr).call(). This will make then call but it seems to return an python list with the values of the listings struct from the contract. The challengeID is in index 4 l[4], which is 0 as expected.

That seems to have worked. I uploaded a new ipynb (version 2) to the same gist, though GitHub isn’t rendering it for me properly right now.

I get output like this for all the challenges to date (and some invalid values top and bottom):

for challengeID in range(10):
    print(cvl.functions.challenges(challengeID).call())
[0, '0x0000000000000000000000000000000000000000', False, 0, 0]
[373276072225334878979, '0x17918Cff28025059e15cEc69910Bc0662e94696B', True, 5000000000000000000000, 102277490000000000000000]
[1403136403455656613885, '0x5E3E872aDb9266adCc47A32758b9329b235963E8', True, 7500000000000000000000, 126313990000000000000000]
[177271168219167682318, '0xF1C5e806d642F7E2f73a679Ba430A5a61D58318B', True, 5000000000000000000000, 26152000000000000000000]
[15064416691696182456, '0xF1C5e806d642F7E2f73a679Ba430A5a61D58318B', True, 5000000000000000000000, 2025480000000000000000]
[1945613519490114942929, '0x14B9c0B60665D221c8Aa53CfF5C25e0EF6F7dA23', True, 5000000000000000000000, 17927490000000000000000]
[69300185130732208524, '0xF1C5e806d642F7E2f73a679Ba430A5a61D58318B', True, 5000000000000000000000, 2624510000000000000000]
[2500000000000000000000, '0x14B9c0B60665D221c8Aa53CfF5C25e0EF6F7dA23', False, 5000000000000000000000, 0]
[0, '0x0000000000000000000000000000000000000000', False, 0, 0]
[0, '0x0000000000000000000000000000000000000000', False, 0, 0]

So what does that mean?