Writing Efficient Rule-Driven Software in Python.notes

TITLE OF PAPER: Writing Efficient Rule-Driven Software in Python
URL OF PRESENTATION: at pycon.org on presentations page, which I can't find
PRESENTED BY: Phillip J. Eby
REPRESENTING: Telecommunity Consultants

CONFERENCE: PyCon 2005
DATE: 3/24/2005
LOCATION: Grand Ballroom
--------------------------------------------------------------------------

REAL-TIME NOTES / ANNOTATIONS OF THE PAPER:
{If you've contributed, add your name, e-mail & URL at the bottom}

Software grows and become unmaintainable.

Lots of the reasons for this is that the rules embodied in the software change a lot.

He's going to show us a way to have our cake and eat it too - a modular way to have the rules while still maintaining efficiency

There are some options to doing rules: expert systems, forward/backward rule chaining systems, etc - typically not integreated into systems

Predicate Dispatch (Chambers/Chen) - a better way to build rules - a rule system that lets you express the rules in python and generate the rule tree at run time

He's mentioning the talk yesterday about why there are a lot of web frameworks.

The trouble with frameworks is that they're not reusable (whatever that means). T h is is cuz you put your code in a framework, not vice versa. He wants to use "predicate dispatch " to "dissolve" the frameworks right out of your project.

The arbitrary method combination part - adapted for python method resolution order
added support for range/equality comparisions w/hash_binary searches
lazy build-out of decision tree

Collection of rules = generic function
declared with @dispatch.generic()

  @dispatch.generic()
  def hasPermission(self, user, perm, subject):
    """ Does 'user have permission 'perm' for 'subject'?"""

Selects applicable rules
  i.e. those whose predicate is true
Determines which rule bodies to execute
  Usually, the one whose predicate is most specific

How to create a "Do the Right Thing" function - the answer is to create a "generic" function that can be told the rules in a data driven manner that then can change the behaviour at runtime. It selects the applicable rules.

Pr e dicate + B o dy = Rule
@security.hasPermission.when(
  "perm==Staff and isins t ance(subject, Facility)"
  )
def checkStaff(self, user, perm, subject):
  return user in subject.staff or security.Denial("%s is not a member of staff at %s " % (user, subject)

If you have multiple rules that apply how do you figure out which one? What you do is that you take the rule that is most specific. If you are getting rules from a user, they will give you a rule that is the "no no no, not then!" rule - that rule is more specific than the previous rules. The generic dispatch routine will apply the more specific rule.

Most Specific == Logical I m plication
isinstance(x, int) implies isinstance(x, object)
  isinst a nce(x,int) is more specific

(A and B) implies A
  (A and B) is more specific than A

Refues the Temptation to Guess
If no rules apply, raise NoApplicableMehtods
If two rules with identical or overlapping predicates apply, raise AmbiguousMethod
  Not a common issue in practice, unless accidental code is duplicated

Implication Detection isn 't (Quite) Perfect
e.g doesn't know that 'x is None' impli e s 'not x'
  so if one rule uses 'x is None' and one uses 'not x' they will conflict when both apply

Method Combination
  Usually you just want the most-specific rule
  lots of not-so-usual cases:
    more-specific rule needs to modify return value of a less-specific rule
    result compbination

Standard method Combination
  most specific rule runs first
  can chain to less-specific rules if i has "next_method" as it's very first argument
  CLOS-style Before/After/Around meethods can also be declared

Custom Method Combination
class Product:
  @dispatch.generic(pricing)
  def getPrice(product, cust=None, opts=()):
    """get this product's price"""
  @getPrice.add_when(default)
  def __basePrice(product, cust, opts):
    """I n clude product's base price"""
    return product.base_price

Specifying Method Cobination
@Product.getPrice.add_when(
  "'blue suede' in opts"
)
def blueSuedeUpcharge(product, cust, opts):
  return 24.95

@Product.getPrice.discount_when(
"cust=='Elvis' and 'blue suede' in opts and product is shows"
)
def ElvisDiscount(product, cust, opts):
  return .1

Decentralized Rules -> Modular Extensibility

Add new operations for existing types
Add new types to existing operations
Change systme behavious by importing modules with additional rules
Allows base system to remain the same, but still have custom behaviour for different users

Q. How do you debug a program that uses rules
A. You may have noticed that I used very specific names for each rule so that you can find the rule that could need further inspection during the debug process.

Q. Can you say the URLs again
A. The presentation is available on the PyCon web site

Q. Why did you decide use the decorators for the data
A. The predicates get compiled into an object structure so that the system doesn't have to be computed more than once and they are available via indexes and even thru an API

Q. Could he imagine combining this system with Pychinko
A. Pychinko is a forward chaining system and that is declaritive and this is all operationally focused

Q. Why are the decoarated used as they are
A. He uses the python compiler to compile the rules into hash code to store for later use

Q. Can you put your own functions into the strings in the rules
A. yes, you can put valid Python code into it with the exception of things that are not evaluated immediately and also lambdas

Q. Can you add a rule that is routed at runtime
A. Yes you can, all of the rules are "added" at runtime because the rule compilation process is done at runtime when the decorators are compiled. There are other ways to build rules at runtime you can use a class api to build rules to pass into the engine and you don't want to parse a string.

Q. Are there order significance like in Prolog
A. no, unless you want it to be, unless you use the Custom Combination method where you can imply order

--------------------------------------------------------------------------
REFERENCES: {as documents / sites are referenced add them below}

Slides are going to be available on the web, he's talking way to fast to follow the slides

http://peak.telecommunity.com/PyProtocols.html

See also Phillip's kicking PEAK framework: http://peak.telecommunity.com/.

His blog is at http://dirtsimple.org/.

--------------------------------------------------------------------------
QUOTES:

--------------------------------------------------------------------------
CONTRIBUTORS: {add your name, e-mail address and URL below}
Erik Rose <corp@grinchcentral.com>

--------------------------------------------------------------------------
E-MAIL BOUNCEBACK: {add your e-mail address separated by commas }

--------------------------------------------------------------------------
NOTES ON / KEY TO THIS TEMPLATE:
A headline (like a field in a database) will be CAPITALISED
    This differentiates from the text that follows
A variable that you can change will be surrounded by _underscores_
    Spaces in variables are also replaced with under_scores
    This allows people to select the whole variable with a simple double-click
A tool-tip is lower case and surrounded by {curly brackets / parentheses}
    These supply helpful contextual information.

--------------------------------------------------------------------------
Copyright shared between all the participants unless otherwise stated...