Saturday, August 7, 2010

The Expectation of Lack of Privacy in Python

One feature of Python that I thankfully rarely have to think of is the whole private, protected, public meme that seems to always popup in programming languages. Programmers coming from other programming languages always want to know how they can make an object's properties private.

Well, besides that fact that black boxes are only good for figuring out what went wrong after the crash. I'd have to say that the whole privacy meme in programming stems from the same sort of administrator == all powerful == programmer way of thinking. Responsibilities are not entitlements. Responsibilities are work.

Here is an example I wrote to clear up what I felt was a miscommunication about using the underscore to create private variables in Python:

Here is a short example I wrote that demonstrates my point about property getters and setters. I have tested cutting and pasting the following into an interactive python 2.5.2 session launched by typing python on the command line without any parameters. The # commented lines should match the output line above it. I hope this clarifies any miscommunication we may have had.

class One(object):
  def __init__(self, value):
    self._x = value
  def getx(self):
    return self._x ** 2
  def setx(self, value):
    self._x = value + 1
  x = property(getx, setx)

class Two(object):
  def __init__(self, value):
    self._my_x = value
  def getx(self):
    """ directly accessing a "private" member of another class """
    return self._my_x._x
  def setx(self, value):
    """ directly setting a "private" member of another class """
    self._my_x._x = value + 2
  # Not defining a setter for the polite_x property makes it read only
  @property
  def polite_x(self):
    """ using the exposed property of another class """
    return self._my_x.x
  x = property(getx, setx)

one = One(10)
two = Two(one)
one._x
#10

one.x
#100

one.x = 10
one._x
#11

one.x
#121

two._my_x._x
#11

two._my_x.x
#121

two.x
#11

two.polite_x
#121

two.x = 10
two.polite_x
#144

two.x
#12

one._x
#12

two.polite_x = 1
#Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
#AttributeError: can't set attribute

I realized 30 seconds after I sent the above example. By looking up the meaning of the underscore and double underscore. That the communication problem was about the underscore as opposed to the double underscore. I was talking about the single underscore because I was asked about the underscore and was only told underscore as feedback when I tried to clarify by asking whether they were referring to the double or single underscore. Which should have been my first clue that I may have my answer judged by someone that didn't fully understand the question they were asking at the time. Which is a big no no. Unless you want to be seen as incompetent because you are being judged by someone who doesn't understand the questions they are asking. And may not be able to understand the answers to those questions. I am loath to admit it but I was just as culpable in this misunderstanding as my understanding of the double underscore was incomplete at that moment. As my mistaken derision of the double underscore came from the accessibility of members like __repr__ and __str__ and __dict__ and __len__. Class members with a double underscore on both sides of the name are system methods and are not meant or implied to being private at all. My lack of interest in making any of my code private was probably the real source of my having no real interest in reading past the disclaimer in the Python documentation which starts with:


9.6 Private Variables
“Private” instance variables that cannot be accessed except from inside an object, don’t exist in Python.


Thus my second email missive of the morning. Making my point all over again using the double underscore. Luckily :). Thank you benevolent dictator for thumbing your nose at private variables!

The private members rewritten with double underscores make those members seem enforceably private. This is because of the exception raised when trying to access those members outside of object and class methods. But this is a misnomer as leading double underscores are actually a name mangling scheme used to hide those members from name collisions with child classes. They can still be accessed directly.

Here is the same example written with "private" members that generate the same results and allows the same access. Despite the miscommunication of apparent privacy. This is because the double underscore's real purpose is for predictable name mangling not privacy.

class One(object):
  def __init__(self, value):
    self.__x = value
  def getx(self):
    return self.__x ** 2
  def setx(self, value):
    self.__x = value + 1
  x = property(getx, setx)

class Two(object):
  def __init__(self, value):
     self.__my_x = value
  def getx(self):
    """ directly accessing a "private" member of another class """
    return self.__my_x._One__x
  def setx(self, value):
    """ directly setting a "private" member of another class """
    self.__my_x._One__x = value + 2
  # Not defining a setter for the polite_x property makes it read only
  @property
  def polite_x(self):
    """ using the exposed property of another class """
    return self.__my_x.x
  x = property(getx, setx)

one = One(10)
two = Two(one)
one._One__x
#10

one.x
#100

one.x = 10
one._One__x
#11

one.x
#121

two._Two__my_x._One__x
#11

two._Two__my_x.x
#121

two.x
#11

two.polite_x
#121

two.x = 10
two.polite_x
#144

two.x
#12

one._One__x
#12

two.polite_x = 1
#Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
#AttributeError: can't set attribute

Friday, August 6, 2010

Bit Flipping in Python

In Python you can access bits directly with the &, |, ^, ~, <<, >> operators on integer objects.

An example bit transformation might be one that moves all of the set bits in a 12 bit number to the left and all of the unset bits in the same 12 bit number to the right.

For example the 12 bit number 101101000001 would be transformed into the 12 bit number 111110000000 using that transformation.

This example bit manipulation can be concisely written in Python as:

n1 = int("101101000001", 2)
bc = sum(1 for i in range(12) if n1 & (1 << i))
n2 = int(("1" * bc) + ("0" * (12 - bc)))

In Python 2.6 and later this example can more concisely be written as:

n1 = 0b101101000001
bc = sum(1 for i in range(12) if n1 & (1 << i))
n2 = sum(1 << i for i in range(11, (11 - bc), -1))

The generic 12 bit manipulation can be more re-usably written as:

# The 12 bits
b12 = tuple(1 << i for i in range(11, -1, -1))

# There are only 13 solutions
result_numbers = tuple(sum(b12[i] for i in range(c)) for c in range(13))
result_strings = tuple(("1" * i) + ("0" * (12 - i)) for i in range(13))

original_number = int("101101000001", 2)
bit_count = sum(1 for i in range(12) if original_number & b12[i])
transformed_number = result_numbers[bit_count]
print result_strings[bit_count]

The Python 2.6 and later version is:

# The 12 bits
b12 = tuple(1 << i for i in range(11, -1, -1))

# There are only 13 solutions
result_numbers = tuple(sum(b12[i] for i in range(c)) for c in range(13))
result_strings = tuple(bin(i) for i in result_numbers)

original_number = 0b101101000001
bit_count = sum(1 for i in range(12) if original_number & b12[i])
# Updated 2011-01-08
# in python 2.7 and later the above can be further optimized
# using range(original_number.bit_length) in place of range(12)
transformed_number = result_numbers[bit_count]
print result_strings[bit_count]

There is no need to define a function call or set of function calls. Bit manipulation easily lends itself to mapping list indexes to result sets that can be prebuilt to aid in readability and execution speed.