Using Iron Ruby or Iron Python as host languages for DSLs

What about IronRuby and IronPython They are CLR implementations of languages that have already proven to be suited for building DSLs. I have two major issues with using these host languages for my DSLs. The first is that, compared to Boo, they don't offer enough control over the resulting language. The second is that both languages run on the Dynamic Language Runtime, which is a layer on top of the CLR, which is what Boo runs on. This means that calling into IronRuby or IronPython code from C...

Creating tests for a DSL

The Message-Routing DSL that we created in chapter 5 routes messages to the appropriate handlers. Listing 8.1 shows a simple scenario using this DSL. Listing 8.1 A simple example using the Message-Routing DSL HandleWith RoutingTestHandler lines return NewOrderMessage( 15, NewOrder, lines.ToArray(OrderLine) ) When I start to test a DSL, I like to write tests similar to the one in listing 8.2. Figure 8.1 A typical structure for a DSL Listing 8.2 The first DSL test usually verifies that the script...

DSL code generation

We've taken DSL code, parsed it, and made it graphical. That was the hard part. The easy part is going the other way, from the abstract concept down to the DSL code. At least, that's how it might seem at first. Let's take a look at our options for code generation and the benefits each approach buys us. The CodeDOM API is the official API that the CLR provides for generating code without being dependant on a particular language. Using this API, we can build an AST and generate the code in a...

Listing 125 An example of the Order Processing DSL

With that knowledge, let's take on our first challenge. Users want to know which rules took active part in the processing of a specific order. If there is a problem, that would make it much easier to track down. Users would know what rules had executed, in what order, and what the result of that execution was. With that feature in mind, let's go about implementing it. When I built the Order-Processing DSL, I choose to use when, rather than if, for the common conditional. That's because when...

Listing 32 Using a DSL to define a task to execute

Task warn if website is down every 3.Minutes() starting now notify admin example.org, server down As you can see, the DSL code doesn't have to work to make the compiler happy, nor does it have the ugly lambda declaration in the middle of the code or all the syntactic baggage (parentheses). It would be difficult to take anything away from the DSL example without losing some meaning. There's little noise there, and noise reduction is important when we're talking about language-oriented...

Ubiquitous language and DSLs

Ubiquitous language is a term used in DDD to describe the way we talk about the software. The ubiquitous language is a spoken language that's structured around the domain model and is used by all members of the team when talking about the domain. A ubiquitous language isn't a DSL, and a DSL isn't a ubiquitous language. A ubiquitous language is used to make communication clearer. Terms from the ubiquitous language are then used in the code of the system. A DSL, on the other hand, can be seen as...

DSL versioning in the real world

In the real world, there are much harsher constraints than I have outlined so far. The DSL changes and business implications described in section 9.5.2 are only a few of the constraints you'll have to deal with when you release your DSL into the wild and need to support new versions. In this section, we'll look at the versioning strategies of three different DSLs, all of them used in the wild by third parties, and at how each has dealt with versioning over time. Brail is a text-templating...

A3 Comments

A comment in Boo is text that the Boo parser ignores it is only there for the enlightenment of the programmer. A comment can begin with a character or a double slash ( ) and continues to the end of the line. Here are a few examples AgeStr prompt(What is your age ) Age int.Parse(AgeStr) convert the string to an integer InSeconds Age * 365.25 * 24 * 60 * 60 print You are InSeconds seconds old. A comment can also begin with a slash followed by an asterisk ( *) and end with the reverse (* ) making...

Testing the DSL scripts

Considering the typical scenarios for using a DSL (providing a policy, defining rules, making decisions, driving the application, and so on), you need to have tests in place to verify that the scripts do what you think they do. In fact, because DSLs are used to define high-level application behavior, it's essential to be aware of what the scripts are doing and protect ourselves from accidental changes. We'll explore two ways to do this, using standard unit tests and creating a fullblown...

Listing 317 Executing a Scheduling DSL script

DslFactory factory new DslFactory() SchedulingDslEngine()) get the DSL instance BaseScheduler scheduler This is where we run the code from the DSL file scheduler.Prepare() Run the prepared scheduler scheduler.Run() First, we initialize the DslFactory, and then create and register a DslEngine for the specific base type we want. Note that you should only do this once, probably during the startup of the application. This usually means in the Main method in console and Windows applications, and in...

The IQuackFu interface

If it walks like a duck and it quacks like a duck, then it must be an IQuackFu. Other languages call it duck typing or method missing (or message not understood) and many dynamic languages support it. Because Boo is a statically typed language (unless you explicitly tell the compiler that you want late-bound semantics ), and because method missing is such a nice concept to have, Boo includes the IQuackFu interface to introduce this capability. The IQuackFu interface, shown in figure 2.1, gives...

Understanding a DSL infrastructure

Before we get into Rhino DSL, let's consider what we want from a DSL infrastructure, and why we need one in the first place. There is a set of problems that we need to deal with and resolve in order to build production-quality DSLs, including the following Dealing with the compiler directly is awkward. It involves a fair amount of work, which needs to be done for each DSL you build. Many DSL implementations share common idioms (as discussed in chapters 4 and 5), and there is little sense in...

Using a business DSL requires business knowledge

This is something that people often overlook. When we evaluate the readability of a DSL, we often make the mistake of determining how readable it is to the layperson. A business DSL uses business language, which can be completely opaque to a layperson. I have no idea what it means to adjust a claim, but presumably it makes sense to someone in the insurance business, and it's certainly something I would expect to see in a DSL targeted at solving a problem in the insurance world. Why wouldn't a...

A2 The Boo interactive shell interpreter and compiler

For many of our examples, we will use booish, the Boo interactive shell. It acts much like a Boo-powered calculator, which is very convenient when you wish to quickly try out some ideas. You start the Boo interactive shell by just entering booish at the command-line prompt. (If you're using Windows, you will first need to open a command-line window.) When booish is ready for you to type something in, it displays three greater-than signs as a prompt. After the prompt, you can calculate 2 + 2....

Why write DSLs

Why do you need a DSL After all, you're reading this book, so you already know how to program. Can't you use normal programming languages to do the job, perhaps with a dash of fluent interfaces and domain-driven design to make the code easier to read So far, I've focused entirely on the how, which is somewhat hypocritical of me, because this entire book is going to focus on abstracting the how. Let's look at the why the different needs that lead the drive toward a DSL. There are several reasons...

Compiler steps

At the beginning of this chapter, we looked at the compiler pipeline and saw that it was composed of a lot of steps executed in order to create the final executable assembly. Most modern compilers work in this fashion,3 but most modern compilers don't 3 I have quite a bit of respect for older compilers, which had to do everything in a single pass. There was some amazing coding involved in them. boast an extensible architecture. Because Boo does have an extensible compiler architecture, you...

Creating a graphical representation for a textual DSL

The idea of creating a graphical representation for a textual DSL seems nonsensical at first, but doing so can be useful. Consider class diagrams in Visual Studio, or having designer and code views of the same web page. A common problem with graphical representations is that they are, by their very nature, a high-level form of communication. This makes them extremely useful when we want to talk at that high level, but not so helpful in many DSL scenarios, because they tend to hide too much....

Creating an IDE for a DSL

An IDE can make or break your DSL in terms of acceptance. Regardless of anything else, having an IDE available implies a high level of dedication to this DSL, and thus a high- quality product. As it turns out, creating an IDE is not that difficult. We'll look at several off-the-shelf components, but before we do, take a look at figure 10.1. It shows a trivial implementation of syntax highlighting for the Quote-Generation DSL. You probably won't be able to see the difference all that clearly in...

AST macros

Meta-methods and AST macros differ in several ways An AST macro has full access to the compiler context and the full AST of the code, and it can collaborate with other macros, compiler steps, and AST attributes to produce the final result. A meta-method can only affect what was passed to it via its parameters. Meta-methods can produce the same results as AST macros, but not as easily. An AST macro can't return values, but a meta-method can. An AST macro is exposed by importing its namespace a...

Integrating an IDE with a DSL application

The main difference between creating an IDE and integrating an IDE lies in the capabilities that you provide the user with. In the IDE scenario, you're providing the user with the tools to do development. In the integration scenario, you're allowing the user to work with the DSL. This is an important difference, because it has big implications for your target audience. You would give an IDE to developers, but that isn't an appropriate tool to give to non-developers, and it doesn't demo as well...

Listing 111 The Quote Generation DSL file structure

This is the documentation header, where we document what this file is doing. A specification refers to a single module specification moduleName A specification denotes actions about requires anotherModuleName, reason for requiring module same_machine_as anotherModuleName We can have multiple specifications in a single file specification anotherModuleName Like in the previous example, you need to explain the structure how things are partitioned but you don't need to discuss the actual details....

DSL startup costs

One thing that might crop up is the startup cost of the DSL. Compilation does take time, even if the results of the compilation are cached after the first round. For many applications, this isn't an important consideration, but for others, the startup time can be a make-or-break decision regarding DSL usage. For scenarios where startup time is critical, you can retain all the advantages that the DSL offers but also get the best startup speed by precompiling the DSL. We'll look at precompilation...

Applying a DSL in a DDD application

It seems natural, when thinking about DSLs, to add DDD to the mix, doesn't it Figure 3.4 shows a set of DSLs in a domain-driven application. In most applications, you'll have a set of DSLs, each of them targeted at one specific goal. You'll also usually have a DSL facade of some kind that will translate the code-driven API to a more language-oriented API. Figure 3.4 DSLs used in a DDD context Figure 3.4 DSLs used in a DDD context There are quite a few domains where DDD doesn't make sense. In...

Going beyond the limits of the language

Boo is a versatile language, and it can do quite a lot for you. We've spent this entire book playing within the confines of Boo, and it wasn't overly confining. Its flexible syntax can allow you to go a long way. That said, there will be situations in which what you want is beyond the scope of the language. In those cases, I have an alternative solution to creating an external DSL. As I mentioned in chapter 1, Boo is an open language, but so far we've only talked about what you can do within...

Summary

We started this chapter by defining a domain model, and then we built a DSL or three, representing three different approaches to building DSLs. The Message-Routing DSL is an example of an imperative DSL. It's executed to perform some goals. The Authorization DSL is a more declarative example but still mostly imperative . It's executed to produce a value, which is later used. The Quote-Generation DSL is a mostly declarative example. It produces an object graph that's later taken up by the...

Designing the Message Routing DSL

Let's start with the simplest alternative an endpoint that can accept JSON-formatted messages and process them. We'll take a peek at the big picture first, in figure 4.1. We'll start from an external application that sends a JSON message to a given endpoint. This endpoint will take the JSON string, translate it to a JSON object, and pass it to the routing module. The routing module will use a set of DSL scripts to decide how Externalapp I JSON endpoint Message-Routing DSL I Business logic...

Naming conventions and holy wars

The choice of naming conventions can be a heated topic. Some people swear by Pascal case, others think that underscores are the pinnacle of clarity, and others are well aware that camelCase is the way the universe was written. And so on. I won't take sides in this argument. I'll just say that I have found underscores to be useful in reducing the density of some of my DSLs. As usual, you need to make your decision based on the wide variety of factors in your own scenario. I urge you to consider...

Naming conventions

The CLR comes with clear guidelines about naming, and the BCL follows them closely. Names are in Pascal case for classes and members, and camelCase for parameters abbreviations should be Pascal case unless they're two letters long, in which case both letters are capitalized, and so on. After a while, this style becomes an ingrained habit. But now I'm going to ask you to forget all of that. We aren't building an API to be consumed by other developers, who can ReadPascalCaseStatementsJustFine....

Boo runs on Java as wellsay hello to Boo Jay

Boo is not just a CLR language it's also a JVM language. You can learn more about Boo on Java in the BooJay discussion group http groups.google.com group boojay . This screencast will introduce you to what BooJay is capable of http blogs. suited for building DSLs. That it runs natively on the Common Language Runtime CLR the technical name for the .NET platform is a huge plus, because it means that your existing toolset is still relevant. I dislike pigeonholing myself, but I'll readily admit...

Listing 210 Manually calling IQuackFu methods

For person as XmlObject in doc.QuackGet Person print When the compiler finds that it can't resolve a method or a property in the usual way, it then checks whether the type implements the IQuackFu interface. If it does, the compiler translates the original method call into a method call using QuackGet, QuackSet, or QuackInvoke. This means that we get to decide what will happen at runtime, which allows us to do some nice things. The XmlObject example is just one of the possibilities....

The structure of Rhino DSL

Rhino DSL is composed of two important classes, both of which you've probably familiarized yourself with by now DslEngine and DslFactory. You can see both of them in figure 7.1. The DslFactory is the external facing interface, used by clients of the DSL, and it uses the DslEngine to perform all the DSL-specific work. The DslEngine is an abstract class that DSL authors are expected to derive from. It provides the standard services out of the box, but it allows you to override most of them as...

Implicit Base Class

The Implicit Base Class is one approach to building a DSL. With this approach, we define a base class in the application code, and then a compiler step in Boo will turn the DSL script into a class that's derived from the defined base class. Hence the base class moniker. The implicit part of the name comes from the fact that there is no reference to the class in the DSL script itself it's implicit. There are three major advantages to this approach. The first is that we can refer to DSL script...

The Rhino DSL project

The Rhino DSL project is a set of components that turned out to be useful across many DSL implementations. It contains classes to aid in building a DSL engine, implicit base classes, multifile DSLs, and so on. We're going to use Rhino DSL throughout this book it's an open source project, licensed under the BSD license, which means that you can use it freely in any type of application or scenario. We're also going to spend chapter 7 dissecting Rhino DSL, to ensure that you understand how it...

Rhino ETL

I built this DSL after I'd had enough of using an ETL tool that wasn't top-notch, to say the least. That ETL tool also used a graphical DSL as the building block, and the pain of using it on a day-to-day basis is one of the main reasons I dislike graphical DSLs. NOTE ETL stands for extract, transform, and load. It's a generic term for moving data around in a data warehouse. You extract the data from one location, transform it probably with data from additional locations , and load it into its...

Exploring the Authorization DSL design

This time, we'll start from the application and move to the DSL, rather than the other way around. We'll start with the Authorization class, which is the gateway to our security infrastructure. It's shown in figure 4.3. The Authorization class contains two methods one for checking an operation, and the second for checking an operation on an entity. The WhyAllowed method lets you retrieve the reason for a certain operation being allowed or denied. One thing to note is that the IsAllowed methods...

Should the User Guide contain the syntax for

That's a good question, and it usually depends on how technical your target audience is expected to be. In almost all cases, I would include the syntax for common operations such as if, unless, and for statements. Those are too useful to keep from your users. Error handling try, except, and ensure , resource management using , and things of that nature should generally stay out of the DSL scripts, and I would avoid pointing them out. Boo also has the notion of statement modifiers, and I...

Listing 1222 Finding references to definitions and replacing them with values

Public class ReplaceDefinitionsWithExpression AbstractTransformerCompilerStep public defines this.defines defines public override void Run Visit CompileUnit Define define defines.FirstOrDefault x gt x.Name node.Name This is a pretty straightforward implementation. This compiler step will search for all reference expressions the names of things, such as variables, methods, types, and so on . If it finds a reference expression whose name matches the name of a definition, it will replace the...

About the cover illustration

The figure on the cover of DSLs in Boo is captioned Le Dauber, which means art student. The illustration is taken from a 19th-century edition of Sylvain Marechal's four-volume compendium of regional dress customs published in France. Each illustration is finely drawn and colored by hand. The rich variety of Marechal's collection reminds us vividly of how culturally apart the world's towns and regions were just 200 years ago. Isolated from each other, people spoke different dialects and...

Listing 316 The implementation of Scheduling DslEngine

Public class SchedulingDslEngine DslEngine protected override void CustomizeCompiler BooCompiler compiler, CompilerPipeline pipeline, string urls new ImplicitBaseClassCompilerStep typeof BaseScheduler , Prepare, default namespace imports As you can see, it doesn't do much, but what it does do is interesting. The method is called CustomizeCompiler, and you're going to learn a whole lot more about customizing the compiler in chapter 4. For now, keep in mind that Boo allows you to move code around...

Listing 1314 The Customer PolicyDsl Engine brings everything together

Public class CustomerPolicyDslEngine DslEngine public CustomerPolicyDslEngine Define defines protected override void CustomizeCompiler BooCompiler compiler, CompilerPipeline pipeline, string urls new ImplicitBaseClassCompilerStep typeof AbstractCustomerPolicy , Prepare, Chapter13.DSL, Chapter13.Model' pipeline.Insert 2, new TreatmentOfToMethodCall defines pipeline.Insert 3, new AddFileNameProperty pipeline.InsertBefore typeof new There isn't much here that's special. It's all fairly common...

Codifying DSL idioms

Most DSL idioms are widely useful across many types of DSLs. The Implicit Base Class pattern, for example, is useful for nearly all DSLs. But after covering the common ground, most DSLs take wildly differing paths. Rhino DSL contains six reusable idioms at the time of this writing, at least . You're probably familiar with most of them by now These six common idioms are the ones I've found most useful across many DSL implementations. Let's look at them each in turn. The good old Implicit Base...