Contents
All the configuration for Deliverance goes in an XML file. This file can configure page matches, transformation rules, proxying, and server settings. Though it is all in one file, there are several major sections. A quick overview:
Everything goes in a <ruleset> tag.
This is the core of what Deliverance does: take a theme along with your content, and apply transformations.
The <theme> element defines the theme you’ll be using. The theme is given as a URL. The basic form is:
<theme href="/theme.html" />
This defines the theme as being at /theme.html. If possible it will be fetched with an internal request, though external requests are also possible (if you host the theme outside of Deliverance).
If you have <theme> at the global level (in the <ruleset> element) then that is the default theme. You can also put it inside <rule> elements, and then it will apply just to that rule set. This is useful if you are using match and page classes.
The theme element also supports pyref Python references. If you use:
<theme pyref="mymodule:get_theme" />
Then define a function like:
def get_theme(request, response, log):
return "/%s/theme.html" % request.host
The request and response objects are WebOb objects.
The default function name is get_theme.
The <rule> element defines a set of transformations. It also supports page classes and request/response matching, which you can read about in those sections. Also, as mentioned above, you can include a <theme> element. But the most important thing is the transformation actions.
The transformation actions are applied in order. The starting point for the transformation is the theme document, and the actions copy elements from the content into the theme.
There are four actions:
Each rule depends on selecting elements from the theme and content. The most basic selection is done with CSS 3 selectors. For instance, this places the element in the content with the id content after the element in the theme with the id header:
<append content="#content" theme="#header" />
You can also use XPath selectors. Any selector starting with / is treated as an XPath expression, while everything else is treated as CSS. CSS can only select elements, and while XPath can select text or attributes Deliverance is only interested in elements. Moving elements around has some limitations, so there are different explicit types of selection:
You can apply any of these like content="children:#content". Not all combinations make sense, and some are not allowed. For instance, <replace content="attributes:#content" theme="elements:#header" /> does not make sense, as you can’t replace elements with attributes. Generally elements: and children: work together, attributes: only works with attributes:, and tag: only works with tag:.
When selecting elements you can use the || operator. This applies to both CSS and XPath selectors, and with the operator you can mix the two. The || operator takes the results of the first selector that matches anything. So content="#content || children:body" will take the element #content if there is one, and if there is not one it will take all the children of <body>. You can mix elements: and children: using ||, though no other types can be mixed like this.
The <replace> action replaces something in the theme with something in the content. Exactly what is replaced depends on the selection type. Some examples:
<replace content="children:#content-wrapper" theme="children:#content" />
this replaces the elements inside the theme element #content with the elements inside the content element #content-wrapper. The resulting document won’t have any element with the id #content-wrapper (unless the theme already had an element with that id).
<replace content="elements:#content-wrapper" theme="elements:#content" />
<replace content="#content-wrapper" theme="#content" />
both of these are the same (elements: is the default selection type). This replaces the theme element #content with the content element #content-wrapper. The resulting document has no element #content.
<replace content="attributes:body" theme="attributes:body" />
this removes all the attributes (e.g., class, onload) from the <body> element in the theme, and moves over the attributes from the content body element.
<replace content="tag:#content" theme="tag:#content" />
this replaces the tag #content in the theme with its corresponding tag from the theme. They might not be the same tag name (e.g., the theme might be a <p> and the content <div>), and all attributes will be taken from the content.
These actions obvious are very similar; <append> puts things from te content after things in the theme, and <prepend> puts things from the content before things in the theme. Some examples:
<append content="children:#sidenav" theme="children:#sidebar" />
this moves the children of the content element #sidenav to the end of the theme element #sidebar, combining the navigation of the theme and content. If you wanted the content navigation to go first you’d use:
<prepend content="children:#sidenav" theme="children:#sidebar" />
Another example:
<append content="children:#sidenav" theme="ol.menulinks" />
this moves the children of #sidenav after the element with the element in the theme <ol class="menulinks">.
<append content="li.reference" theme="children:ol.menulinks" />
this moves any element in the content like <li class="reference"> into the element in the theme <ol class="menulinks">.
You can also use these with attributes:
<append content="attributes:div#content" theme="attributes:div#content-wrapper" />
This adds any attributes from div#content into the theme element div#content-wrapper – but when the attribute already exists in the theme, the theme attribute is kept. If you use <prepend> then when there are overlapping attributes the content attribute value is kept.
You can’t use tag: with these actions.
The <drop> action is used to remove problematic elements from a theme or content. You can also use it with if-content for other kinds of conditions. Since this action doesn’t move elements around you don’t need to provide both content and theme attributes; either will do.
A common example is removing a stylesheet which introduces conflicts:
<drop content="link[href $= '/sitestyle.css']" />
This is one of the more advanced selectors that CSS 3 allows. You can’t (confidently) use it in browsers yet, but you can in Deliverance! The $= operator means ends-with, so this selector could be described as: all link elements with href attributes that end with ‘/sitestyle.css’. Other operators are the obvious =, ^= which means starts-with, *= which means contains. Note that all comparisons are case-sensitive.
Another example:
<drop content="attributes(class):a.external-link" />
which removes the class from any <a class="external-link"> elements.
<drop content="tag:font" />
which removes all <font> tags, but doesn’t remove any actual text content.
All the actions can take an attribute if-content="selector". This selector is attempted on the content, and if it doesn’t match anything then the action is skipped.
Except for <drop>, in actions it is necessary that the theme selector match exactly one element. If it doesn’t match any element, then the action can’t be performed. If it matches many elements then the action is ambiguous: which element should be the target? Also, when no content element is matched it is a sign something is wrong, and in some cases if multiple content elements are matched it is a problem.
The default handling for all these is to log the problem at the level “warn”. If there are multiple matches when that is not expected, the default handling is to log and use the first element. There are four attributes to override this:
A value notheme="ignore" means that the action is ignored and only a “debug” level logging message is produced. A value like nocontent="abort" means that if there is no content then all theming will be aborted, and the unthemed content page will be displayed. For manytheme and manycontent you can indicate first or last to select the first (default) or last element. You can combine this like manytheme="ignore:first".
An example:
<replace content="children:#content" theme="children:#content"
nocontent="abort" />
Almost all rules should have at least one action with nocontent="abort". This is the action that moves the primary body of the content into the theme. If that primary body can’t be found then when people browse the page they’ll see the theme and lose all the useful content.
manycontent is used when you are using selection types attributes: or tag: – in both these cases the theme and content element match up one-to-one.
When using <drop> multiple theme and content elements are expected, so no error handling is necessary in that case.
In addition to manipulating the theme and content documents, you can bring in content from a third source using the href attribute on a rule.
If you include href the document at that location is used instead of normal content document. For example:
<append href="/sidebar" content="children:body"
theme="children:#sidebar" />
This appends all the elements in the body of /sidebar into the theme. This can be used to make the theme more dynamic. Deliverance doesn’t produce content on its own, or modify content in complex ways, but you can use includes like these to introduce several sources of dynamic content into a single page: /sidebar can be a dynamically generated page itself.
By default actions move elements from the content to the theme. That is, if you select content with an action, later actions won’t be able to access those content elements. This is particularly useful in cases like this:
<body>
<div class="navigation">links...</div>
Some content
</body>
with the rules:
<rule>
<replace content="children:#navigation" theme="children:#sidebar" />
<replace content="children:body" theme="children:#content"
nocontent="abort" />
</rule>
If elements were copied the navigation would show up twice in the resulting page. If you do want content copied instead of moved then add move="0" to the action.
There are several actions that are “standard”. These are common for HTML generally. The actions are:
<rule>
<replace content="children:/html/head/title"
theme="children:/html/head/title" nocontent="ignore" />
<append content="elements:/html/head/link"
theme="children:/html/head" nocontent="ignore" />
<append content="elements:/html/head/script"
theme="children:/html/head" nocontent="ignore" />
<append content="elements:/html/head/style"
theme="children:/html/head" nocontent="ignore" />
</rule>
These copy over the title and any links, scripts, or stylesheets from the content into the theme. These are always applied unless you use <rule suppress-standard="1">.
Note: for a simple site you can mostly ignore this, and just define a single <rule> that is applied to all requests. But in many cases you will need to apply specific rules only to parts of the website. For example, different applications may have navigation in specific parts of the page, and you want rules to move that navigation around the page that is specific to the application.
Often different sets of rules will apply to different parts of the site. To define what rules go with what requests/responses there is the concept of “page classes”. These are classes that apply to the page, and any rules with those classes are used. For example, imagine a page has the classes trac and default. Then these rules would be used:
<rule class="trac">...</rule>
<rule class="default">...</rule>
Like CSS any rules that match any of the classes on a page will be run. Also like CSS you can have multiple classes anywhere, separated by spaces. Unlike CSS the classes are ordered, and define the order the rules are run in.
There are several ways to define classes on a page. If you have control of the server you can add the response header X-Deliverance-Page-Class with the page classes. You can also add a class attribute to proxy elements.
A third way is with <match>. These elements match requests and add classes. You can use them like:
<match domain="lists.*" class="lists" />
This will add the “lists” class to any requests going to a domain lists.*. The request matching is described in request/response matching.
Deliverance comes with a proxy, deliverance-proxy. This starts a server and proxies HTTP requests to other backend servers (Zope, Apache/PHP, paster, etc).
Settings for the proxying also go in the rule file. The <server-settings> element defines aspects of the server, while <proxy> defines specific servers to proxy to.
This element looks like:
<server-settings>
<server>localhost:8080</server>
<execute-pyref>false</execute-pyref>
<display-local-files>false</display-local-files>
<dev-allow>
127.0.0.1
192.168.0.1/24
</dev-allow>
<dev-deny>
192.168.0.121
</dev-deny>
<dev-htpasswd>/etc/deliverance-dev-users.htpasswd</dev-htpasswd>
<dev-user username="bob" password="uncle" />
<dev-expiration>60</dev-expiration>
</server-settings>
You can’t actually use quite all of these together. Going over the individual settings:
The <dev-*> tags define access to the developer console.
The <proxy> element is what defines what gets routed where. The basic pattern is:
<proxy path="/trac">
<dest href="http://localhost:10002" />
</proxy>
This proxies any requests to /trac to the dest address. A request to something like /trac/view/1 will go to http://localhost:10002/view/1. If you wanted to keep /trac you can use <proxy strip-script-name="0">.
If you want to pass the Host header through without changing it, use <proxy keep-host="1">. This generally represents the request accurately, but has not been the norm for systems like this, and often the original host name is only available in X-Forwarded-Host. “Real” proxies like Squid usually preserve the Host header. Apache does not unless you configure it to do so. So if you have setup your system for Apache proxying you probably weren’t preserving the Host header.
You can also match against things besides the path; anything in request/response matching is available. But without a simple path match the path stripping won’t work. (A path match like path="regex:.*/manage$" wouldn’t work, for instance.)
You can also add page classes for the proxied request, by using <proxy class="page-class">.
You can use <proxy editable="1"> to allow the files references by the proxy to be editable through the developer debugging console. If you do this you have to have a <dest> that references a file:///... URL.
The proxy can contain several elements...
The <dest> element defines the destination. You must have a dest element.
You can give an href value of both http://... URLs and file:///... URLs. Files are served directly without proxying, though this is seemless to the rest of the process.
The value in href can be a URI template (though only the simplest form of template). You can use headers like {Host}, environmental variables like {REMOTE_USER}, or the variable {here} which points to the directory containing the rule file. I.e., if the rule file is in /etc/deliverance/rules/ruleset.xml then {here} would be file:///etc/deliverance/rules. This template is substituted for every request, so it can be fully dynamic.
In addition to this you can use pyref Python references, like:
<proxy path="/trac">
<dest pyref="mymodule:get_proxy_dest" />
</proxy>
with the function:
def get_proxy_dest(request, log):
if not request.remote_addr.startswith('192'):
raise AbortProxy('Bad remote_addr: %r' % request.remote_addr)
return 'http://localhost:10002'
You can return any URL. URI template substitution is not performed. If you raise AbortProxy then the <proxy> will be skipped, and another proxy will be looked for. (If nothing matches a 404 error is returned.)
If you want to perform request rewriting (as described in the next section) you can use <dest next="1"> and the request modifications will be performed but otherwise the proxy section will be skipped.
This modifies the request before it is proxied on. You can add headers and run Python code against the request.
To add a header:
<request header="X-Project-Name" content="My Project" />
You can also use a pyref Python reference, like:
<request pyref="mymodule:modify_proxy_request" />
with the code:
def modify_proxy_request(request, log):
request.header['X-Project-Name'] = request.host.split('.')[0]
return request
You can modify the request in-place or return a new request.
This modifies the response. Like the request you can add headers:
<response header="Cache-Control" content="max-age=0" />
You can also rewrite links, which means you can proxy from a site that doesn’t particularly want to be proxied. So if you proxy to http://othersite.com then of course all the links it returns will be to http://othersite.com. Using link rewriting it will rewrite all those links to go back through the proxy. This includes the Location header, any references to domain in cookies, and all links in the HTML.
<response rewrite-links="1" />
And of course there’s pyref:
<response pyref="mymodule:modify_proxy_response" />
with the code:
def modify_proxy_response(
request, response, orig_base, proxied_base, proxied_url, log):
# orig_base: the original URL base: http://localhost:8080/trac
# proxied_base: where dest sent it to: http://localhost:10001/
# proxied_url: the full destination, e.g.,
# http://localhost:10001/view/1
# request.url: the full original URL, e.g.,
# http://localhost:8080/trac/view/1
response.body += 'look at me!'
return response
Many elements can match requests and responses: the match element, rule, and proxy. They all use the same matching.
There are several attributes that match against different parts of the request or response. If you provide multiple attribute then they must all match.
These attributes match a string value against a pattern you provide. There are several kinds of patterns; different attributes default to different types of patterns, whatever is most logical for the attribute value.
First, the attributes:
Each pattern string can start with patterntype: which defines what kind of pattern it is. Here’s a list of the available patterns (these also apply to the key/value matching described in the next section):
There are other matches that match both a key and a value, like matching headers and environmental keys. These look like environ="REMOTE_USER: bob", like key: pattern. The key can be a wildcard, and if any key that matches that wildcard matches the pattern then it is a match. Only wildcards are allowed for the key. All patterns default to exact:.
The attributes available:
You can also use a pyref, with a function like:
def match_request(request, response, response_headers, log):
if response.headers.get('x-notheme'):
raise AbortTheme
return True
Note that response_headers is a list of all the headers, including <meta http-equiv> headers.
pyref attribute references allow you to hook into Python code. To read about the details see the pyref document.
You must be logged in to view the developer console. The login page is available at /.deliverance/login – you define the users using the server-settings configuration.
After you log in you can add ?deliv_log to any page URL to get a log at the bottom of the page. The log describes in detail how the page was transformed, including information about how the proxy was chosen and any subrequests. It also lets you browse the source involved, see what the selectors select in the content or theme, or get a list of interesting ids and classes in the content.
There is an experimental feature that allows client-side theming using Javascript. This serves the blank theme to the user, with content pulled in via XMLHttpRequests. This means the user will get a response quickly, seeing the contentless theme page to start with (and so you should write the theme so that any slots contain text like “Loading...”). The individual chunks that are pulled in will be cachable on the client side.
To enable this put this in your config:
<clientside />
You may also put conditionals on it, like <clientside path="/slow-part" />. You can only use one theme in this case, as page classes are not matched before you send the theme. Also any rules that use href="..." must be possible to apply out-of-order, as the XMLHttpRequest’s may come in unexpected orders. Using, for instance, <replace> and <append> on the same element is problematic in this case – a better solution would be to clear out the theme target element and use <prepend> and <append>.
Deliverance will be careful about using this technique. Only clients that are known to run Javascript will be sent these pages (this is tested by setting a cookie via Javascript, and then testing on subsequent requests if the cookie is set). Also, pages won’t be themed like this until it’s known that the backend request yields HTML. If these aren’t true the normal server-side theming is used, which should be equivalent.
This has only been tested on Firefox.