『 读书笔记 』effective Python

1. Pythonic Thinking

1.1 know which version of python you’re using

1.2 follow the pep 8 style guide

1.3 know the differences between bytes, str and unicode

1.4 wirte helper functions instead of complex expressions

1.5 know how to slice sequence

1.6 avoid using start, end and stride in a single slice

1.7 use list comprehension instead of map and filter

1.8 avoid more than two expressions in list comprehension

1.9 consider generator for large list comprehension

1.10 prefer enumerate over range

for index, item in enumerate(list):
    print 'index : {}, item : {}'.format(index, item)

1.11 use zip to process iterators in parallel

1.12 avoid else blocks after for and while loops

1.13 take advantage of each block in try/except/else/finally

2. functions

2.1 prefering exceptions to return none

2.2 know how closures interactive with variable scope

In [1]: def sort_priority(values, group):
   ...:         def helper(x):
   ...:                 if x in group:
   ...:                         return (0, x)
   ...:                 return (1, x)
   ...:         values.sort(key=helper)
   ...:

In [2]: values = [8, 3, 1, 2, 5, 4, 7, 6]

In [3]: group = [2, 3, 5, 7]

In [4]: sort_priority(values, group)

In [5]: values
Out[5]: [2, 3, 5, 7, 1, 4, 6, 8]

2.3 consider generators instead of returning list

2.4 be defensive when iterating over arguments

2.5 reduce visual noise with variable positional arguments

2.6 provide optional behavior with keyword argument

2.7 use none and docstrings to specify dynamic default arguments

# when the function is defined, default argument values are evaluated just once per module at the loading time.

def log(msg, time=datetime.now()):
    print 'msg: {}, time: {}'.format(msg, time)

def log_version(msg, time=None):
    msg_time = time if time else datetime.now()
    print 'msg: {}, time: {}'.format(msg, msg_time)

2.8 enforce clarity with keyword only argument

3. Classes and Inheritance

3.1 prefer helper classes over bookkeeping with dictionaries and tuples

3.2 accept functions for simple interfaces instead of classes

3.3 use @classmethod polymorphism to construct objects generically

3.4 initialized parent classes with super

3.5 use multiple inheritance only for mix-in utility classes

3.6 prefer public attributes over private ones

3.7 inherit from collections.abc for custom container types

4. metaclasses and attributes

4.1 use plain attributes instead of get and set methods

4.2 consider @property instead of refactoring attributes

4.3 use descriptors for reusable @property methods

4.4 use __getattr__, __getattribute__, and __setattr__ for lazy attributes

4.5 validate subclass with metaclass

4.6 register class existence with metaclasses

4.7 annotate class attributes with metaclasses

5. concurrenc and parallelism

5.1 use subprocess to manage child processes

5.2 use threads for blocking i/o, avoid for parallelism

5.3 use lock to prevent data races in threads

5.4 use queue to coordinate work between threads

5.5 consider coroutines to run many functions concurrently

5.6 consider concurrent.futures for true parallelism

6. built-in modules

6.1 define functions decorators with functools.wraps

6.2 consider contextlib and with statements for reusable try/finally behavior

6.3 make pickle reliable with copyreg

the pickle module's serialization format is unsafe by design. the serialzed data contains what is essentially a program that describes how to reconstruct the original python object. this means a malicious pickle payload could be used to compromise any part of the python program that attempts to deserialize it.

6.4 use datetime instead of time for local clocks

6.5 use built-in algorithms and data structures

6.6 use decimal when precising is paramount

6.7 know where to find community-built modules

7. collaboration

7.1 write docstring for every function, class and module

7.2 use packages to organize modules and provide stable apis

7.3 define a root exception to insulate callers from apis

7.4 know how to break circular dependecies

# dialog.py
import app

class Dialog():
    def __init__(self):
        pass

save_dialog = Dialog(app.prefs.get("hello"))

def show():
    pass

# app.py
import dialog

class Prefs():
    def get(self, name):
        pass

prefs = Prefs()
dialog.show()

then if you use app.py in your project, you’ll absolutely get an error like bellow:

In [1]: import app
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-f8537898a049> in <module>()
----> 1 import app

/Users/chenshan/Desktop/app.py in <module>()
----> 1 import dialog
      2
      3 # import ipdb; ipdb.set_trace()
      4
      5 class Prefs():

/Users/chenshan/Desktop/dialog.py in <module>()
      8         pass
      9
---> 10 save_dialog = Dialog(app.prefs.get("hello"))
     11
     12 def show():

AttributeError: 'module' object has no attribute 'prefs'

the problem with a circular dependency is that the attributes of a module aren’t defined until the code for those attributes has executed(after step 5), but the module can be loaded with the import statement immediately after it’s inserted into sys.modules(after step 4)

in the example above,

  1. the app.py import dialog before doing anything
  2. then, dialog.py import app firstly, currently the imported app is just an empty module
  3. then, in dailog.py, the save_dialog = Dialog(app.prefs.get("hello")), at that time, we’re sure that there are really a module named app exists in current runtime, but it’s an empty module, so will raise an attribute error like this.

7.5 use virtual env for isonated and reproducible dependences

8. Production

8.1 consider module-scoped code to configure deployment env

8.2 use repr string for debugging output

8.3 test everything with unittest

8.4 consider interactive debugging with pdb

8.5 profile before optimizing

8.6 use tracemalloc to understand memory usage and leaks