CSP Templates

CSP is Civilians own type-safe and high-performance template system. It uses a Turing complete expression language called Java. Time to learn: 15-30 minutes.

Introduction

Civilian includes an own template system called CSP, shortcut for Civilian Server Pages.
For a introduction into Civilians template philosophy please consult the Templates chapter.

The steps to use CSP in your application are the following:

  1. You create a template file – by convention it has the extension .csp – and write your template in CSP syntax.
  2. You then invoke the CSP Compiler which generates a Java class derived from Template. The compiler translates the CSP syntax into a series of Java commands. This Java class needs to be included in your IDE project or build process.
  3. When you want to produce response content (e.g. in a Controller method) you instantiate this Java class. Dynamic data needed by the template is passed to its constructor. Then you simply call its print(Writer) method or pass the template object to Response.writeTemplate(Template).
    At runtime the generated Java template class is used, the CSP file is only needed during development time.
  4. Whenever you change the CSP file, you repeat the compilation process.
Of course, different CSP templates are created for different response outputs. Usually each Controller will have an own CSP template.

The CSP approach has the following important advantages:

  • There is no runtime interpretation of template syntax with all its pitfalls.
  • Compiled templates give you all the benefit of compile time checks. Runtime exceptions due to incongruencies between your templates and the rest of your code base just vanish. Any code change which breaks the template will be spotted by a Java compiler error in the generated template class.
  • Output is fast since it runs as compiled Java code at JVM speed and does not need an interpreting template engine.
  • There is no dependency on external libraries, such as taglibs in JSP.
  • You can use all Java design techniques to compose templates.
  • You are not constrained by a simplistic template expression language but can use all the expressiveness of Java.
  • You don't need to prepare artificial temporary models which can be digested by the templates expression language. CSP can directly work with any Java object.
The rest of the chapter describes how to write CSP templates. This is a rather lengthy description, so we hope you make it to the end (to become a lifetime CSP ninja).

CSP syntax – the basic idea

CSP – like most other templates systems – is based on the following very simple idea: A template describes the static scaffold of a document, intermingled with embedded code snippets. These snippets are responsible to insert dynamic data into the scaffold, to produce the actual document:
Hello <%name%>!
Here we expect to find the literal "Hello " in the output. The following <% switches from literal mode to expression mode, terminated by %>. name is expected to be a variable or expression defined in the execution context of the template. At runtime it will be replaced by its value. Then again we expect a literal !. The output could therefore be
Hello Wilma!
Of course one needs more than simple variable replacement: We need expressions, conditional output, loops and more. Many template systems invent an own expression and control flow syntax for this. CSP does not - its expression language is plain Java. Congratulations - zero learning curve ahead.

CSP is compiled into Java code

The CSP compiler takes your CSP file and turns it into a Java class. At runtime only this generated class is used.
In the above example the compiler would produce a class similar to:
package com.myapp.greeting;

import org.civilian.template.Template;

public class GreetingTemplate extends Template
{
    public GreetingTemplate(String name)
    {
        this.name = name;
    }
    
    protected void print() throws Exception
    {
    	out.print("Hello ");
    	out.print(name);
    	out.println("!");
    }
    
    protected String name;
 }
Even if the generated code is nicely formatted, we really never look at it, we just use it, e.g. in a Controller. to produce a response. (And we never ever edit this generated class, only the CSP file).
Note that GreetingTemplate is derived from Template (out is defined in Template), and therefore we can generate Response content in the following way:
Response response = ...;
GreetingTemplate t = new GreetingTemplate("Wilma");
response.writeTemplate(t);
and on the wire the greetings are.

In-depth walkthrough of CSP syntax

A CSP file has four sections:
  1. Template configuration (optional)
  2. Template declaration
  3. Template body
  4. Additional template functions (optional)
A CSP template containing all four sections looks like:
import org.example.data.Person;
template(Person person)
{{
Hello <%getFullName(person)%>!
}}

private String getFullName(Person person)
{
return person.getFirstName() + ' ' + person.getLastName();
}
We now walk through these parts, examining one syntax construct at a time. This example shows how the whole template file developed in this tutorial might look like.

1. Template configuration

The template configuration consists of a sequence of instructions, each being optional. The CspCompiler translates most of them into Java constructs, like package or import statements, class javadoc and class annotations. An instruction always covers a single line and consists of a keyword followed by whitespace followed by a value. The instructions must be specified in the following order.

The encoding instruction allows you to set the encoding of the template file (The default is UTF-8). Alternatively you can also use parameters on the CSP compiler to specify the encoding.

encoding ISO-8859-1
The package instruction allows to set the package of the generated class. A closing semicolon is optional. The CSP compiler can deduce the package from the location of the CSP file, so most of the time we will skip this instruction.
package com.myapp.greeting
If your template uses classes from packages different than the template package, you need to add import instructions. Again a closing semicolon is optional:
import java.util.List;
import com.myapp.util.GreetingHelper
As you already guessed, the package and import instructions are simpley turned into Java package and import statements.
With all the people inventing new languages left and right we couldn't resist to tinker with the Java import statement and allow relativ imports, increasing immunity against package refactorings:
import ./flowers.Flower   // -> com.myapp.greeting.flowers.Flower
import ../util/StringUtil // -> com.myapp.util.StringUtil
The prolog instruction allows you to generate lines before the class declaration. You may want to use it to define a Javadoc comment or class level annotations
prolog /**
prolog * What a template.
prolog */
prolog @SuppressWarnings("unused")

2. Template declaration

The Template declaration is used to generate the declaration of the template Java class and its constructor. Remember, all runtime data used by the template is passed to its constructor.
The template declaration consists of keywords, sometimes followed by parameters or parameter lists. The declaration may be placed in a single line or in multiple lines.

The declaration starts with the only required keyword. Checkpoint reached!

template
Do you want to pass parameters to your template? If yes, open a bracket, specify the parameters, and close with a bracket. Think of constructor parameters.
(String name, List<Flower> flowers, GreetingHelper helper)
By default the generated class is public. To turn it into a non public class continue with
package-access
To generate an abstract templates class (e.g. to be used as base class for other templates) add:
abstract
By default the generated template class is automatically derived from Template. Want another parent class? Maybe you prepared a base template class in your application and want to inherit all its goodies. Then use the extends keyword (and don't forget to include a import instruction if needed).
extends MyBaseTemplate
Want no parent class? (This topic motivates why you would want to do this). Write:
extends -
Want the template to implement an interface? Use the implements keyword followed by a list of interfaces. Again use either qualified names or add imports to the template configuration.
implements Adorable, BattleNotificationSender
Next you declare the mixins available in your template. (Mixins will be explained in detail in the next section): The mixin keyword is followed by a list of mixin classes. The HtmlMixin, LangMixin, TableMixin and FormTableMixin mixins offered by Civilian can be simply specified as "html", "lang", "table" and "formTable".
mixin html, lang, MyMixin
The mixins are available in the template as fields, whose field names are derived from the Mixin classes. You can explicitly state this field name if you want a different one. Here is one for people raised up in MS territory:
mixin html:m_html, lang:m_lang, MyMixin:m_mymixin
No declaration without exceptions. By default templates follow the Civilian design rule to simply declare that they might throw an Exception (even if they don't intend to). You want to tame that? Overrride with the throws keyword
throws IOException, IllegalArgumentException
or for the optimistic
throws -
Putting it all together the declaration might look like:
template(String name, List<Flower> flowers, GreetingHelper helper)
package-access
abstract
extends MyBaseTemplate
implements Adorable, BattleNotificationSender
mixin html, lang, MyMixin
throws IOException, IllegalArgumentException
Looks complicated but that was really the full treatment, the minimum declaration is simply
template 
You might miss a way to specify the name of the generated class. Glad that you asked: The name is just taken from the CSP file. So if the file is called GreetingTemplate.csp we end up with a class named GreetingTemplate.java. (Yes, the suffix "Template" is pure convention).

That's all about template declarations. With the configuration and declaration we defined the class level stuff and the constructor of the template class.
Time to produce some output. So on to the hard stuff and best part.

3. Template body

As already said the CSP file is turned into a Java class. This class will have a print-method which will produce the template output when called. The template body will just define the content of this method.
You start the template body after the template declaration on a new line with
{{
For the sake of symmetry you will end the template body with
}}
The similarity to method blocks is intended. But the double braces actually turn on what is called the literal template mode. Now by default everything in the template body is treated as literal and written as such to the output.

Static content
The following is a complete CSP template, which when run always writes out the constant string Hello World

template
{{
Hello World
}}
Of course this also works with angle brackets:
template
{{
<html>
<body>Hello World</body>
</html>
}}

Embedded expressions
You might appreciate this simple beauty but probably still want to have some dynamic action? Remember the first look at the syntax, which introduced embedded expressions, using PHP/ASP/JSP-like <% %> brackets. The opening <% starts an embedded code snippet, ended by the closing %> which must be in the same line as the opening part. Here we go:

template(String name)
{{
Hello <%name%>!
}}
This will print the value of the name variable passed to the template.
(In the following examples we mostly only show the inner content of a template body, skipping template declaration and config).

What can be placed inside <% %>? Any Java expression, e.g a variable reference, an arithmetic expression, a method call which returns a non-void result.

1 + 2 = <%1 + 2%>
SPAM GREETINGS <%name.toUpperCase()%>
You can insert as much embedded expressions in a line as you want:
Hey <%name%>, 1 + 2 = <%1 + 2%>
Now what is actually printed when we execute the template? Let's take a look at the generated code:
out.print("Hey ");
out.print(name);
out.print(", 1 + 2 = ");
out.println(1 + 2);
The field out is a TemplateWriter inherited from the Template base class. TemplateWriter in the end is just a java.io.PrintWriter. Therefore all values are finally serialized to strings calling value.toString() or String.valueOf(value) before they are written to the output. So it only makes sense to print simple objects like Strings or primitive values. If you print a fancy Customer object it will just simply fallback to its toString() method. Escaping text or formatting values is another topic. Later in the Mixins section we will learn how to achieve that.

Embedded statements
Besides expressions you can also place one or more statements in a <% %> snippet. The statements need to be terminated by a ; like in Java. In the following example we call a method named doSomething, passing the name:

Hey <% doSomething(name); %>!
In the generated template class this translates to
out.print("Hey ");
doSomething(name);
out.println("!");
Why you want to do this? Of course because the called method prints something to the output (having access to the TemplateWriter or obtaining it as parameter) or produces some side-effect. Again the Mixins provide good use cases for this technique. Here is a brief glimpse, using the HtmlMixin to escape a string and write it to the output:
Hey <% html.text(name); %>

The transcript operator @
Expressions and method calls, nice and sweet. What about CSPs control flow capabilities? Time for the @-Operator.
Let's test your intuition again – can you spot the meaning of:

@if (name.length() > 20)
howdy big <%name%>
@else
howdy shorty <%name%>
This compiles to:
if (name.length() > 20)
{
out.print("howdy big ");
out.println(name);
}
else
{
out.println("howdy shorty ");
out.println(name);
}
Got it? A line in literal mode which begins with a @-character is just copied into the Java class (minus the @).

Let's use the same technique to demonstrate a loop:

@for (int i=0; i<name.length(); i++)
The <%i%>-th character in your name is <%name.charAt(i)%>
No taglib consultant was tortured to implement this example. In the end it just translates to plain Java.

In the last examples we silently used an important feature of CSP, namely indentation.
Indentend lines after a @-line (as demonstrated in the previous two examples) are put into a Java block – except that we don't need curly braces: Kudos Python!
Else indented lines are also indented in the ouput, exactly what we want for things like HTML output.

<body>
<h1>Hi</h1>
</body>
The result is pretty printed HTML code of pure handcraft grade. JSP output with only rough similarity to the template was yesterday. Don't believe this? Take a look at the output of the Civilian samples.

The indent technique comes at a price: Use only tabs or spaces to indent your lines. But be consistent, else the CSP compiler will complain.

The component operator [
The [-operator is somewhat arcane but nonetheless powerful. The TableMixin makes heavy use of this operator, so please study this section for an motivating example.
The component-operator addresses the following problem: Sometimes you want to embed content into structures which are highly repetitive. Instead of explicitly repeat these structures in the template, a ComponentBuilder is asked to print output before and after content is printed, therefore enabling much shorter templates.
The [ character at the beginning of a line is used to turn on the component-operator. It can be used in a single-lined form

[<cb-expression>] <content>
or in a multi-lined form
[<cb-expression>
<content>
]
<content> can be any CSP template content. <cb-expression> is a Java expression which returns an instance of ComponentBuilder. The operator content will be printed, preceeded and followed by output defined by the ComponentBuilder.

Now we have only some syntactic sugar to cover, namely <%%>, <%/%>, <%?%> and super-calls.

Using <%%>
What if a line output deliberately starts with whitespace? Indentation rules could cause trouble. An empty <%%> at the start of the line disambiguates between indent and whitespace content:

<div>
Hey <%name%>
<%%> what's up?
</div>
In the same way you can enforce whitespace at the end of a line. Another use case is if your literal line starts with an operator character:
hello  <%%>
<%%>@...
<%%>[...

Using <%/%>
Literal lines always produce a linebreak in the output (whereas @-lines never do). But there are situations where we don't want this behaviour. Use a <%/%> at the end of a literal line to prevent a linebreak in the output:

Hey <%/%>
@if (name.length() > 0)
big <%/%>
<%name%>

Using <%?%>
The <%?%> syntax allows you to produce a inline conditional version of the last example. A first <%?%>-block contains a boolean Java condition, the second <%?%>-block terminates the conditional content. The content in between is only printed if the condition evaluates to true. Rewriting the last example gives

Hey <%? name.length() > 0 %>big <%?%><%name%>

Using @super
Templates can be derived from other templates. If the super-template itself defines constructor-parameters then the derived template must pass values to the super-constructor. You can do this by placing a super-call in the first line of the template body

template(String name) extends BaseTemplate
{{
@super(name, true);
...
Now you know all about CSP literal mode. Piece of cake.

4. Additional template functions

We just ended the template body with closing double curly braces. This could end the template file – or we through in some more, namely additional methods and/or fields. Look at this example:
template(int x, int y)
{{
    @calcArea();
    The area of <%x%> times <%y%> is <%printArea();%>.
}}

private void printArea()
{{
     <%area%>!
}}

private void calcArea()
{
    area = x * y;
}

private int area;
Lots of things to cover:
The template got an additional private field named area.
We defined a private helper method calcArea(). It does not use literal template mode (note the simple braces). It accesses the template parameters x and y which are visible throughout the template.
We defined a private helper method printArea(). This one uses literal template mode again and when called adds to the output.
To summarize: CSP allows to define helper methods, either pure computational or again using CSP literal syntax. Helper variables can be defined within a method or on a class level.
What do you think?

Template Mixins

We can think of a lot of useful helper methods when writing templates: HTML related methods like escaping text or localization methods to format numbers and date values. Since Civilian templates can inherit from base templates inheritance would be an obvious option to provide such useful helper methods.
But this would easily lead to bloated base classes. Mixins to the rescue! We would like to include helper methods (defined in some mixin) into our template when needed.
The only bad thing about mixins is that Java does not support them. Did not bother us much, and we invented some sort of replacement. Repeating an example from above, template mixins allow to write CSP code like this:
Hey <%html.text(name);%>, 100 * 321 is <%lang.format(100*321)%>
The first embedded expression calls a method html.text(String). It returns void and therefore we end the call with a semicolon. This method takes the parameter, escapes any character which is not allowed in HTML text content, and writes the result to the output.
Similar, the second embedded expression calls a method lang.format(int). This method formats the integer parameter according the locale of the response and returns the formated value (which is then printed to the output).
Now, html and lang are just fields in the generated template class with access to the TemplateWriter.
These fields are not available by default but must explicitly be defined. The syntax section already explained how to do that:
template ... mixin html, lang ...
{{
...
The names html and lang refer to the predefined HtmlMixin and LangMixin. Please take a look at their Javadoc to see in detail what methods they provide.
But of course you can also build your own mixin implementations and reuse them in your templates.
template ... mixin com.myapp.template.BarMixin:foo
{{
Hello <%foo.howdie();%>
...

The Table mixin

The TableMixin allows to easily build complex HTML-tables. Since it also implements ComponentBuilder it can be used in combination with the component-operator [.
The basic idea of the TableMixin is to first define a column layout. The layout defines the number of columns in the table and may specify widths or HTML attributes for each column or gap column. For example:
[50]5[30%, align=left][class='help red']
defines a layout with these columns:
  1. a column with width 50px
  2. a gap column with width 5px
  3. a left aligned column with width 30%
  4. a column with the class attribute set to "help red"
and then just output a series of cell contents while the TableMixin will keep track of surrounding <tr> and <td> elements and print appropriate output. Here we use the component operator [ to instruct the TableMixin that we are starting a new cell. The template
template mixin table:t
{{
@t.columns("[]5[30%, align=left, class='info']");
@t.startTable("class", "table table-border");
[t.rowspan(2)] a
[t] b
[t] c
@t.endTable();
}}
would produce the following output
<table class="table table-border">
<colgroup>
<col>
<col width="5px">
<col width="30%">
</colgroup>
<tr>
<td>a</td>
<td></td>
<td class="info" align="left">b</td>
</tr>
<tr>
<td></td>
<td class="info" align="left">c</td>
</tr>
</table>

The CSP compiler

The CSP compiler turns a CSP file in a Java template class.
It is implemented by class org.civilian.tool.csp.CspCompiler. It prints a detailed help message when run without any arguments.
CspCompiler expects an input file or directory and can optional parameters:
java org.civilian.tool.csp.CspCompiler param* input
The input is either a single CSP file or a directory. In the later case all CSP files in the directory are compiled, by default also recursing into subdirectories.

One of the -out:* parameters must be used to the determine the location of the output file:
-out:input writes the output file into the same directory as the input. This is the default setting.
-out:package <directory> writes the output file into the package directory below the given root directory

... -out:package src/main/generated ...
-out:dir <directory> writes the output file into the given directory.
-out:file <file> writes the output file to the given file.

The compiler only generates the output class if the timestamp of the input file is greater than that of the output file. The force option turns this check off:

... -force ...

The -enc, -enc:in, -enc:out parameters allow you to specificy encodings of input and/or output, if they are not UTF-8:

... -enc ISO-8859-1 ...      // for both input and output
... -enc:in UTF-16 ... // for the input
... -enc:out ISO-88592-2 ... // for the output

The -v parameter defines verbosity of compiler messages. Use a value of 0, 1 or 2 for the level.

... -v level ...

Normally the CSP compiler only accepts files with extension ".csp". If you want another extension, write

... -ext myextension ...

Project organisation and IDE support

Where do you put CSP files and generated templates classes in your IDE project? Since templates are usually used by a single controller, we typically put the CSP file and the generated template class into the same package as the controller.
For the Maven minded the CSP file will go under resources and the template class under generated, but we typically put it into the same package folder, as seen in the samples.
Anyway just instruct the CSP compiler where to pick up CSP files and where to put the output.

The CSP compiler needs to be run every time you change a CSP file. Of course you can run an ant script (like the one generated by the scaffold tool). In Eclipse we usually run the CSP compiler as External Tool. Here are the steps to configure such a call:

  1. Create new External Tool Configuration
  2. Enter the path to java.exe in Main/Location.
  3. Enter in Main/Arguments:
    -cp <path to civilian.jar>
    org.civilian.tool.csp.CspCompiler
    -force
    -v 1
    ${resource_loc}
    Of course you can tweak the arguments to your needs, e.g. adding an -out parameter.
    The ${resource_loc} variable means that the CSP compiler acts on the active Eclipse resource. If you opened a CSP file, then this file will be compiled. If you selected a package in the project tree, the CSP compiler will recursively compile all its CSP files.
  4. Configure the Refresh panel: After the CSP was compiled we want Eclipse to compile the generated template Java class. Since we usually put the generated class in the same folder as the CSP file, we just check "The folder containing the selected resource" and Eclipse will pick up the changes. If you have a different file structure, choose the right refresh location for your case.
We also define a short cut key for the eclipse command "Run Last Launched External Tool" to quickly launch the CSP compiler. (See Prefrences/Keys). Unfortunately Eclipse does not allow to define a short cut for a specific launch command. Other IDEs will have similiar or better possibilies to run a Java command from the IDE.
Now we edit the CSP file, save, press the shortcut-key and the Java template class is ready.

Syntax coloring would also be nice, but right now is beyong the scope of Civilian. A current possibility is to instruct the IDE to open CSP files in a JSP or HTML editor window for at least some coloring support.

CSP as standalone template system

When you don't instruct otherwise the generated template class is derived from Template and operates on a TemplateWriter. This tightly binds the template to Civilians request processing context.

But CSP can simply be turned into a standalone template system: If you use the extends instruction to not derive from Template

template ... extends - ...
then the generated toplevel class will operate on a TemplateWriter and will have no other dependencies to Civilian classes. The TemplateWriter actually implements the pretty indentation stuff. You construct a TemplateWriter simply by passing a Writer class to its constructor.

Template reuse

Complex applications will need to think about how to efficiently organize templates. If for example multiple HTML pages are served, parts of the HTML like the head or a navigation bar might be relatively constant whereas the content differs from page to page. It would make no sense to repeat constant parts like the HTML head in multiple templates therefore we would like to reuse templates code in other templates.

Template composition

Template objects can be passed to other templates and placed in the output. In the example we can define a PageTemplate which generates the
template(Template content)
{{
    <html>
        <head><title>The App</title><head>
        <body>
            <%content%>
        </body>
    </html>
}}
We can then create Templates for page contents and wrap it into a PageTemplate to generate the whole page
getResponse().writeTemplate(new PageTemplate(new LoginTemplate());

Template inheritance

Templates can derive from other templates. In the HTML example a base PageTemplate may define the basic structure of the HTML page and derived templates fill in the blanks. The PageTemplate might be defined as:
template abstract mixin html
{{
    <html>
        <head><title>The App</title><head>
        <body>
        	@printContent();
        </body>
    </html>
    
    protected abstract void printContent();
}}
and an implementation might look like
template(String name) extends PageTemplate
			
@Override protected void printContent()
{{
    Hello <%html.text(name);%>
}}
Notice that in the derived template the template declaration is not followed by a template method which is in fact inherited from PageTemplate. Instead we implement the abstract method printContent.


At this point you might want to experiment with the syntax and study the compiled output. Please also take a look at the samples which contain a lot of CSP templates. Thank you for your endurance.