ron rothman.ron rothman
selectively conformist

» Python 3 Rounding Surprise

by Ron
No comments yet

I discovered this subtle change when one of my unit tests mysteriously failed once I ported it from Python 2 to Python 3. A computed price was consistently one microdollar higher in the Python 3 version.

I eventually tracked it down to a change in rounding behaviour between Python 2 and 3. According to the docs (emphasis mine):

if two multiples are equally close, rounding is done toward the even choice

I proved it to myself thusly:

$ python2 -c 'print( round(8.5) )'
9.0
$ python3 -c 'print( round(8.5) )'
8

This method of rounding is called “banker’s rounding.” The good news is that banker’s rounding better suits our need for a fair rounding method. That is: our goal in rounding is to have rounding errors even out in aggregate, and banker’s rounding achieves that better than always rounding up (i.e., away from zero).

Revert to Python 2 Behaviour

Should you find the need to use Python 2’s rounding algorithm, it’s available as part of the Decimal module as decimal.ROUND_HALF_UP (example).

References

» Python 3 Exception Chaining

by Ron
No comments yet

Exception chaining solves two problems in Python 2.

1. Swallowed Exceptions

If, while handling exception A, exception B inadvertently occurred, exception A was completely lost.

# Python 2
try:
    raise ValueError('original problem')
except Exception as exc:
    4/0  # bad calculation while handling the original problem

Sadly, the original problem is nowhere to be found in our Python 2 stack trace:

Traceback (most recent call last):
  File "/tmp/t1.py", line 4, in <module>
    4/0
ZeroDivisionError: integer division or modulo by zero

2. Preserving Tracebacks

Sometimes it’s useful, when catching an exception, to raise a new exception (e.g., a custom error type). In Python 2, the original exception could only be preserved to the extent that you could explicitly include it in the new exception’s message or args. It’s annoying enough to have to encode the original exception’s message into your new exception; but if you want to pass along its stack trace, then you’re pretty much out of luck. (Unless you want to add several confusing lines to your except block; even then, you’d have to pre-format the traceback in a way that you knew would be appropriate for use downstream. This is not a good situation to find yourself in.)

# Python 2
try:
    # some code that raises an exception
except Exception as exc:
    # the traceback of "exc" will be lost once we raise the following exception
    raise MyErrorType('The flurble operation failed due to {}'.format(exc))

Implicit Exception Chaining

Python implicitly chains any exception that occurs while handling another exception. This solves problem #1:

# Python 3 (but same code as Python 2 version above)
try:
    raise ValueError('original problem')
except Exception as exc:
    4/0  # bad calculation while handling the original problem

Without any modification to the code, note how Python 3 keeps track of the exception chain for us:

Traceback (most recent call last):
  File "/tmp/t1.py", line 2, in <module>
    raise ValueError()
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/t1.py", line 4, in <module>
    4/0
ZeroDivisionError: division by zero

Explicit Exception Chaining

If we explicitly state that some exception was the proximate cause of another, Python 3 will remember the ensuing causal chain for us:

# Python 3
try:
    a = 3
    b = 0
    x = a / b  # uh-oh

except Exception as exc:
    raise ValueError('Some data was bad') from exc

(Note the “from exc” on the last line.)

The chain of exceptions is preserved and displayed, each with its own stack trace:

Traceback (most recent call last):
  File "/tmp/t.py", line 20, in <module>
    x = a / b  # uh-oh
ZeroDivisionError: division by zero

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/t.py", line 23, in <module>
    raise ValueError('Some data was bad') from exc
ValueError: Some data was bad

Happy chaining!

» Safe Password Storage – How Websites Get It Wrong

by Ron
No comments yet

Here’s the recording of my talk “15 minutes w/ Beeswax: Safe Password Storage – How Websites Get It Wrong.”

And here are the slides:

Safe password storage: How Websites Get It Wrong from Ron Rothman

» Python at Scale: Concurrency at Beeswax

by Ron
No comments yet

My presentation from the NYC Python Meetup: Python at Scale; Concurrency at Beeswax.

Concurrent Python at Beeswax – Ron Rothman – NYC Python Meetup 2020 from Ron Rothman

» Python Dependencies The Right Way

by Ron
No comments yet

My BazelCon 2019 lightning talk, Python Dependencies The Right Way*, has been posted. Please excuse my raspy voice; I woke up that morning with a sore throat and drank copious amounts of tea and honey to nurse my voice back to semi-working condition. Enjoy!

» Python 3 f-strings

by Ron
No comments yet

One of my favorite features of Python 3 is f-strings (Formatted String Literals).

Each of these print statements emits the same output. But which of them is the easiest to write and read?

company_name = 'Beeswax'
location = 'New York City'

print('The company {} is based in {}. {} is a great place to work.'.format(company_name, location, location))

print('The company {company_name} is based in {location}. {location} is a great place to work.'.format(company_name=company_name, location=location))

print(f'The company {company_name} is based in {location}. {location} is a great place to work.')

Even if that’s all they did, I’d love f-strings. But they’re much more powerful than that. (For example, you can dereference objects inline: f"Number of connections: {aerospike_client.connection_count}") In fact, you can evaluate arbitrary expressions including function calls within the f-string braces, and all variables in scope can be passed as arguments. See the references below for the details.

References

https://docs.python.org/3.7/reference/lexical_analysis.html#f-strings

» mtwsgi: A Multithreaded Python WSGI Implementation

by Ron
No comments yet

I wanted to combine the simplicity of Python’s built-in WSGI server with the benefits of concurrency, so I wrote this multithreaded drop-in replacement for wsgiref.WSGIServer.

Also included: a server adapter for use with Bottle.

Suggestions welcome.
…read the rest 

» Python 2.7.3 (Bug) Broke My Namedtuple Unpickling

by Ron

I just learned (*ahem* the Hard Way) that CPython 2.7.3 namedtuple is buggy when combined with pickling and older Python versions.

If you pickle a namedtuple using CPython 2.7.3, you won’t be able to successfully unpickle it using 2.7.2 (nor, presumably, using even earlier versions).

Here’s a short example. First, run this with Python 2.7.3:

# Pickle a namedtuple with python 2.7.3

import pickle
from collections import namedtuple
Point = namedtuple('Point', ('x', 'y'))

a = Point(4, 9)

with open('foo', 'w') as f:
    pickle.dump(a, f, -1)

Then, run this with 2.7.2:
…read the rest 

» Cygwin cron: Spurious Messages in Event Viewer

by Ron

I’ve been using Cygwin on my Windows partition to run a particular job every minute. Cron (Vixie’s cron) works perfectly, except that messages like this one were appearing (every minute!) in my Windows event log: …read the rest 

» Fixing minidom.toprettyxml’s Silly Whitespace

by Ron

Python’s xml.dom.minidom.toprettyxml has a feature/flaw that renders it useless for many common applications.
…read the rest 

» 2 Become 3

by Ron
"click me!"
“Leo loves you.”

» It’s a Boy!

by Ron

Please welcome our newest family member, Blake, into the world.
…read the rest 

» Is Home Birth Courageous?

by Ron

It’s funny. I’ve recently been told—by more than one person—that Sara and I are “brave” for planning to deliver our baby at home. This from people who are educated, intelligent and have my respect. And yet, they couldn’t have picked a worse way to characterize the decision. What’s going on here?

Read on, and then you tell me: isn’t it actually more courageous for a low risk mother-to-be to voluntarily walk into a hospital to give birth?
…read the rest 

» Canada Trip: Posted

by Ron

Finally posted the itinerary and photos from our summer trip to Quebec & Ontario.

Activities included:

  • Visiting friends
  • Cycling 100 miles of scenic converted rail line
  • Genealogy research in rural Quebec
  • Consumption of many and varied animal parts (not to mention: chocolate, cheese, poutine, et al.)
  • Ice wine tasting
  • Quebecois Folk music concerts
  • Watching Beluga whales from the breakfast table
  • Niagara falls in a barrel

Canada Trip: PostedCanada Trip: PostedCanada Trip: PostedCanada Trip: PostedCanada Trip: PostedCanada Trip: Posted

» CSA Inventory, Week 10

by Ron

Produce Roundup

fresh fruits and vegetables

This week’s vegetables. (Click for larger view.)

Fresh, organic and local. $12.

This week’s take:
…read the rest 

» Southeast Asia Trip: Epilogue

by Ron

We’re back home. (Boo.)

We’ll post our photos as soon as we sort through [all 1500 of] them.

Update: the photos have been posted.

Southeast Asia Trip: EpilogueSoutheast Asia Trip: Epilogue
Southeast Asia Trip: EpilogueSoutheast Asia Trip: Epilogue