Dancer Advent Calendar 2014

Past and future

Wed, 24 Dec 2014 00:00:00 -0000

Past and future

This last article is meant to reflect on the past, in the present, and speculate about the future, perhaps even - with the cover of darkness - offer some future plans, in quiet murmur, of course. :)

Popularity

We all care about popularity. Popularity isn't just cool factor, it's also a measurement of success - doing well, going in the right direction.

But how do you measure popularity? It's hard.

We have different mechanisms made just to measure success. In our world that would Github stars, Github watchers, and even MetaCPAN votes. However, if you take a look at Moose, it only has 78 stars on Github, and only 15 watchers. Would you say Moose is less successful than Dancer? Not at all. This measurement simply is far from accurate.

However, there is another way to measure popularity.

One of my favorite and most respected authors provides the following comment in all of his projects:

All complex software has bugs lurking in it, and this module is no exception. If you find a bug please either email me, or add the bug to cpan-RT.

Complex software accumulates tickets. These are bugs, feature requests, failing tests, missing tests, questions, and so on. Hopefully authors will collect these in a ticket system like RT or Github.

The amount of tickets you have indicates how many of these came up. While some authors will open a staggering amount of tickets for their projects (which is nothing short of being thorough and responsible) and projects may have a devoted user who opens tickets upon tickets, most tickets are opened by the audience of users. The number indicates how many have used your software to a degree of contacting you, providing input, and found you or your project approachable and tried to help improve it.

You can count the number of tickets for a project, remove the core authors (and check for when they weren't core authors yet - it happens sometimes), normalize to amount of users opening the issues, and look at the numbers.

Dancer has two main repositories: Dancer 1 and Dancer 2. Together they have accumulated around 1,900 tickets. Yes, that's close to two thousand tickets. About 1,000 of them are pull requests. Yes, that's right. Over one thousand pull requests. Actually, my numbers are off. I only took into account Github. Dancer has been running on RT earlier and there's a quite a few there as well. That would also be off, because we accept tickets through any medium. We get them in personal emails, on the mailing list, at conferences, on IRC, and so on.

Beyond all measurements, I'd say we reached people. More importantly, I would say these people reached us. I cannot be more proud of this achievement.

I can talk about Booking using Dancer. I can talk about Shutterstock using Dancer. I can talk about MoonFruit, UK2, Tilt, FinanceJobs, PerlMaven, Geekuni, the Strehler CMS, and even a spiritual church and an anarchist library using Dancer.

I can talk about all of these, but that's not the great part. The great part is our contributors.

Contribution is key

Having someone contributing their time and skill is priceless. Contributors are the life-force of a project - not the core developers. Almost any core developer in Free and Open Source will undoubtedly tell you that development's gratitude is someone using it, thanking you for it, or better yet, improving it further by contributing something - a test, a documentation patch, or even opening tickets.

Except for Dancer's founder, Alexis (@sukria) Sukrieh, all Dancer core developers started as contributors. This includes yours truly and everyone else in the team. This hasn't changed since. We always saw and see ourselves simply as contributors with more responsibility.

Dancer has enjoyed the help of an astonishing amount of contributors. Overall, the Dancer project has received contributions from over 270 people. While this seems big it fails to account for many contributors and many forms of contribution. It's merely the number of recognized code committers.

Sam Batschelet organized the Dancer conference, Gabor Szabo took the Dancer website under his care, Peter Martini provided a Dancer Docker image, David Dick packages the project for Fedora, Paul Cochrane is working on improving the documentation for beginners, and there are numerous other examples for tremendous work done by the community.

Let me take a moment to thank you all for the work you've done and the work that you do, in the community and for the community.

So now, where is all of this heading?

What the future holds...

It's hard to say what the future holds for Dancer, but we do have quite a few plans.

The short list contains:

Thorough documentation overhaul
Improve the plugin system for Dancer2
Port all useful plugins to Dancer2
Rewrite fair portions of the testing suite
Many core cleanups
Additional functionality
Speed optimizations
Compatibility shims
Code linting

We even have some idea regarding features we want to introduce:

Full asynchronous support
Route naming
Chained hooks
Stronger command line tooling
More decoupling of components

As you can see, there are plenty of ideas and the opportunities are abound.

While we want to add some of these features as quickly as possible, we need to make sure we get them right, that we design them well, that we do not overlap with other features, and that we don't have to revert them.

In fact, what we need is your help.

I can promise some will be difficult to implement, some will take considerable time, and some will be like trudging through mud (especially any OS compatibility work), but above all, I can promise you will have a good time. :)

I would like to thank everyone for an amazing year. I hope to see you and work with you this new year and provide you all with another set of articles at the next Advent Calendar.

Have a merry Christmas, a wonderful Hanukkah, and a tremendous Festivus, for the rest of us!

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Dynamic routing in Dancer is dynamic

Tue, 23 Dec 2014 00:00:00 -0000

Dynamic routing in Dancer is dynamic

Dancer provides with a number of ways to do dynamic routing, A.K.A., chained actions. Let's take a look at situations that lead to such a requirement and how to achieve it.

The use-case

Let's say you were building a message board that supported any number of boards - each with a custom URL. For example, if this were something for a Beethoven fan site you might have a board for discussing each genre of work composed by him, with URLs like:

http://www.beethovenfans.com/string-quartets/
http://www.beethovenfans.com/symphonies/
http://www.beethovenfans.com/piano-sonatas/

Visiting the top-level board url would show you a list of all posts under that board. You could start a discussion by simply using /new-post under the board name. So to start a discussion under string-quartets, you'd go to:

http://www.beethovenfans.com/string-quartets/new-post

Similarly, you could view or permalink to a specific post as:

http://www.beethovenfans.com/string-quartets/opus-18-no-6

Where opus-18-no-6 is the unique identifier of the post - possibly generated from the title of the post.

Similarly, to upvote a post on the board:

http://www.beethovenfans.com/string-quartets/opus-18-no-6/upvote

Each time someone accesses one of these URLs, you first need to check if the discussion board exists. To put it in code:

get '/:board' => sub {
    my $board = param('board');

    # validation code for $board
    ...
};

get '/:board/new-post' => sub {
    my $board = param('board');

    # validation code for $board
};

get '/:board/opus-18-no-6/upvote' => sub {
    my $board = param('board');

    # validation code for $board
};

Each of these handlers have to start by validating the existence of a discussion board before they do anything else. It makes sense to have the discussion board validation logic in one place.

Setup method

One way to do this is by defining a special setup method which you call when the application starts up. This method can use the prefix keyword to set up routes for each board.

sub get_boards {
    # in a real life application this would come from a
    # database. for the sake of simplicity, we simply return
    # a hard-coded list

    return qw(string-quartets symphonies piano-sonatas piano-trios);
}

sub setup_routes {
    my @boards = get_boards;

    foreach my $board (@boards) {

        prefix "/$board" => sub {

            # any variables here will be accessible to the handlers
            # defined below
            my $welcome_message = "Welcome to the group for discussing: $board";

            get '/' => sub {
                # show a list of all the posts under the board $board
                # for demo's sake simply return the welcome message
                return $welcome_message;
            };

            get '/:post_id' => sub {
                # do the work here or delegate to a specialised class
                my $post_id = param('post_id');

                # check if the ID is valid and do something with it
                # for demo's sake show variables you have access to:
                return "I was called with board: $board, post id: $post_id"
                     . " and have access to the welcome_message variable: $welcome_message";
            };
        };
    }
}

Then in your app.pl:

#!/usr/bin/env perl

use FindBin;
use lib "$FindBin::Bin/../lib";

use beethoven;
beethoven->setup_routes;
beethoven->to_app;

Calling setup_routes essentially materialises the routes beforehand based on what get_boards returns. If the user enters a board that doesn't exist, it will automatically get picked up by Dancer's default handling of missing routes and a 404 error will be returned to the user.

This approach works well if the number of boards is small. Say in hundreds, or may be in thousands, but not in hundreds of thousands. After a certain point the Dancer process will grow too big in memory. That might or might not be a problem depending on your infrastructure.

Also if boards are not fixed and are created all the time then this approach will not work very well as it would required an app restart to pick up all the new boards (or for that matter changes to the URL slugs for the existing boards, or deletions).

The hook

There is another way that allows us to dynamically check for a board rather than materialising all our routes up front. The basic idea is to combine prefix with a before handler. Let's put together some code to illustrate this idea:

hook before => sub {
    my $board = param('board');

    if ($board && board_is_valid($board)) {
        var board => $board;
    }
};

prefix '/:board' => sub {

    get '/' => sub {
        # show a list of all the posts under the board $board
        # for demo's sake simply return the welcome message
        my $board = var('board') or pass;

        my $welcome_message = "Welcome to the group for discussing: $board";
        return $welcome_message;
    };

    get '/:post_id' => sub {
        my $board = var('board') or pass;

        # do the work here
        my $post_id = param('post_id');

        # check if the ID is valid and do something with it
        # for demo's sake show variables you have access to:
        return "I was called with board: $board, post id: $post_id";
    };
};

sub board_is_valid {
    my $board = shift;

    return if not defined $board;

    # in a real life application we would check this against a table in
    # a database or through a backend service. for the sake of
    # simplicity, let's just check against a hardcoded list
    my %valid_boards = map { $_ => 1 }
                       qw(string-quartets symphonies piano-sonatas piano-trios);

    return defined $valid_boards{$board};
}

This time, our before handler will catch the request first. It sets up the board variable using var that is automatically passed around to each handler. The handler first checks if the board variable exists and if it doesn't, it simply defers the handling to the Dancer's default route handling. Since no other routes to handle the :board exist, this results in a 404.

While it alleviates the need to build all possible routes up front, the one downside to this approach is having to check for the presence of board variable in each handler. On the positive side, you can dynamically add boards without restarting the application for them to be available.

Even more ways

Additional approaches for the dynamic route handling problem exist. Most notably, another approach was recently documented by Yanick Champoux uses megasplat to produce chained actions.

Minimum version

This article requires Dancer2 version 0.159000.

(hopefully released soon enough.)

Conclusion

Dancer has many methods of producing dynamic routes (or "chained actions") and each of them has its benefits and drawbacks.

Author

This article has been written by Deepak Gulati for the Perl Dancer Advent Calendar 2014.

Copyright

Creative Commons Deepak Gulati 2014.

The Dancer community policy

Mon, 22 Dec 2014 00:00:00 -0000

The Dancer community policy

In October 2014 we have introduced our Dancer core and community policy and our Standards of Conduct. You can find them at Dancer::Policy or Dancer2::Policy (identical documents).

While the Dancer community has always pride itself of the warm feeling members have received, helpfulness and kindness were abundant, we still found it necessary to introduce a policy document.

Why do we need it?

While the policy is not an answer to a situation in the Dancer community, it is to behavior we've seen in other communities in and out of the tech industry.

There are people who are not only discriminated against, but also marginalized daily. There are people who feel pushed out of the way, made small and insignificant, and dis-empowered by others who tend to have a more favorable starting point - what is usually referred to as "privileged".

We felt it is absolutely crucial to provide a document that makes it absolutely clear, without a shadow of a doubt, that the Dancer community is maintained and owned by its members - all having a fair and equal stance in it. It is essential that everyone in the community feel and understand this space belongs to them just as much as it does to anyone else.

Introducing a problem where none exists?

So, are we really just fixing a situation that is not broken? If all is fine and well in the Dancer community, why provide such standards of conduct? That's a good question.

Standards of conduct have multiple purposes:

Set expectations of behavior
It is important that people in the community are aware of both what is expected from them (such as "you are expected to not be abusive towards anyone") and what they can expect from others (such as "you can expect not to be abused by anyone"). Setting these expectations allows people to let their guard down, enjoy their time, approach others, and be an active member without fearing abuse.

The community, above all, should be a safe space for all of its members.
Provide a stern warning
The document clearly explains that not only do we have expectations, but we will take measures against those who violate them. This helps defend the expectations we described above.

If you will violate those standards, we will make sure you are held accountable. This makes sure these are not empty promises and that they won't be perceived as such.
Enforcement transparency
While we provide these standards and a promise to enforce them, it is also important to provide transparency. You need to know what will be done and how decisions will be made.

Can I still be me?

Many people worry that with a standards of conduct policy they will not be able to be themselves anymore. This is worth addressing.

The standards of conduct clearly stipulate you may not, under any circumstance, be abusive to others. If violating that robs you of your being, it means you are an abusive person - a person we hope other community members will not have to experience or deal with.

This, however, is not the same as "I will need to watch what I say". Having to make an effort to be kinder, more composed, or in general more respectful, is an understood effort. It is an effort we make every day. We make sure we're more respectful towards our friends, colleagues, peers, and we make sure we curse much less around our parents.

While it is not always a simple task to work on our behavior, it is far different than violating your core principles.

In short, if your core principles are to hurt others, and you feel like our standards of conduct negate that, we've done a good job.

Conclusion

The community is maintained by the community members. Each and every single member should be regarded with respect, and we should all feel a sense of belonging to and in our community.

The Dancer standards of conduct have been introduced to make sure new members understand this and know what we each expect from each other.

We were thrilled with getting such positive results from a community. Members have unequivocally said we are happy to make this message clear to all current and future members.

Thanks

Praise should be given to #p5p (The Perl 5 Porters list) which provided the base for the policy document, to @sungo for pushing to make community standards published clearly and for doing so with the Perl IRC servers, and to ribasushi for helping revise and suggest improvements to the policy as it was being written.

The biggest thanks is, as always, to the wonderful and loving Dancer community and all of its members. :)

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Middlewared Dancer: our usage of middlewares

Sun, 21 Dec 2014 00:00:00 -0000

Middlewared Dancer: our usage of middlewares

Now that we understand what middlewares are and how to use them, we would like to share a small secret with which we ended the previous article.

While much of the existing middlewares provide additional features, there are middlewares that can help supplement a web framework or reduce the amount of work it does.

In Dancer2 we rely heavily on Plack and therefore middlewares provide much-appreciated help which plays well with our framework, since it's all PSGI at the end of the day.

Here are the current middlewares we use internally in Dancer2:

Static

First, as described in a previous article, Dancer2 now uses Plack::Middleware::Static to provide static files.

This helped provide consistency with both the previous behavior (a la Dancer 1) and with how users actually expect static file serving to work.

Head

Using Plack::Middleware::Head, we can provide correct HEAD requests much more easily. A HEAD request should not have a body, and the middleware simply deletes the body of a response if the request method is HEAD.

This allows us to define every GET request as HEAD and have the middleware delete the response body to produce an appropriate response.

One less concern!

ContentLength

Originally Dancer2 would have to set the length of a response content body every time it was set. Although this could have been improved by only setting it at the very end, we would still have to account for three kinds of responses available in PSGI.

Instead, we're using Plack::Middleware::ContentLength to decide and set the Content-Length header for us after we've returned a response. Our code is thus cleaner and less error-prone.

FixMissingBodyInRedirect

Having Plack::Middleware::FixMissingBodyInRedirect provides the proper response body when a user returns a redirect. You don't have to care about it anymore, it's simply there, automagically.

RemoveRedundantBody

When returning a status of 204 or 304 the body needs to be stripped.

Plack::Middleware::RemoveRedundantBody takes care of that for us. Since it uses Plack::Util's status_with_no_entity_body, it also takes care of statuses 100 and 101, which were not covered beforehand.

Conclusion

When writing a web framework, you (or at least we) would like to focus on the features and richness of the DSL and syntax, and not the small details of HTTP implementations.

Using middlewares reduces the amount of work we do to keep up with all of these small bits and does a damn fine job at it.

We will continue to explore new middlewares we can make use of, and perhaps even push some of our code into middlewares so it could be decoupled and even used by non-Dancer code.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Meddling with Middlewares

Sat, 20 Dec 2014 00:00:00 -0000

Meddling with Middlewares

Plack provides a useful component called a Middleware, which wraps applications with functionality.

You can use them to your advantage to tack on additional features and improvements to your application.

Middleware usage

Adding middlewares can be breeze using the Plack::Builder syntax:

use MyApp;
use Plack::Builder;

builder {
    enable 'Auth::Basic', authenticator => sub {
        my ( $user, $pass, $env ) = @_;
        # test $user and $pass somehow
    };

    MyApp->to_app;
};

Since to_app returns an app, you can use it as the last statement to builder. In this example we add Plack::Middleware::Auth::Basic middleware to our new application stack.

We can add more:

builder {
    enable 'Auth::Basic', authenticator => sub { ... };

    enable 'Rewrite', rules => sub {
        return 301
            if s{^/foo/?$}{/bar/}
            or s{^/baz/?$}{/quux/};

        ...
    };

    MyApp->to_app;
};

Now having Plack::Middleware::Rewrite as well. Having it before Auth::Basic will assure the rewrites will happen first. Having it after Auth::Basic will assure authentication will happen first.

Multiple Dancer applications

As explained in our to_app article, you can set up multiple Dancer2 applications using the same syntax:

use MyApp;
use MyApp::Admin;
use Plack::Builder;

builder {
    mount '/'      => MyApp->to_app;
    mount '/admin' => MyApp::Admin->to_app;
};

We can still add middlewares to that stack:

builder {
    enable 'Rewrite', rules => sub {
        ...
    };

    mount '/'      => MyApp->to_app;
    mount '/admin' => MyApp::Admin->to_app;
};

Not bad, but there's one more situation to cover.

Going nuts

Let's take a look at another situation: adding authentication using a middleware:

builder {
    enable 'Auth::Basic', authenticator => sub { ... };

    mount '/'      => MyApp->to_app;
    mount '/admin' => MyApp::Admin->to_app;
};

But wait. This doesn't make sense. We want to only enable authentication for the admin application. This will set it up to both of them.

Since the Dancer application is a callback, and builder wraps callbacks by providing another callback... you might see where this is heading:

builder {
    mount '/'      => MyApp->to_app;
    mount '/admin' => builder {
        enable 'Auth::Basic', authenticator => sub { ... };

        MyApp::Admin->to_app;
    };
};

Pretty cool, huh? Very!

Conclusion

Middlewares are awesome and very easy to integrate with Dancer. In fact, we use middlewares internally in Dancer2, which you will read about in the next article.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Authentication for the masses

Fri, 19 Dec 2014 00:00:00 -0000

Authentication for the masses

Just like at some point every developer writes a templating system, every web developer eventually writes a form for authentication.

It's simple: we check the user credentials on a request and decide whether to continue or redirect them to a form. The form allows them to submit their username and password and we save that and create a session for them so when they now try the original request, we recognize them and allow them in.

Basic application

The application is fairly simple. We have a route that needs authentication, we have a route for showing the login page, and we have a route for posting login information and creating a session.

package MyApp;
use Dancer2;

get '/' => sub {
    session('user')
        or redirect('/login');

    template index => {};
};

get '/login' => sub {
    template login => {};
};

post '/login' => sub {
    my $username  = param('username');
    my $password  = param('password');
    my $redir_url = param('redirect_url') || '/login';

    $username eq 'john' && $password eq 'correcthorsebatterystaple'
        or redirect $redir_url;

    session user => $username;
    redirect $redir_url;
};

Tiny authentication helper

Dancer2::Plugin::Auth::Tiny allows you to abstract away not only the part that checks whether the session exists, but to also generate a redirect with the right path and return URL.

We simply have to define what routes needs a login using Auth::Tiny's needs keyword.

get '/' => needs login => sub {
    template index => {};
};

It creates a proper return URL using uri_for and the address from which the user arrived - something we didn't do ourselves.

We can thus decorate all of our private routes to require authentication in this simple manner. If a user does not have a session, it will automatically forward it to /login, in which we would render a form for the user to send a login request.

Auth::Tiny even provides us with a new parameter, return_url, which we can use to send the user back to their original requested path.

We will still need to handle the password verification, but we have yet another small helper for this.

Password hashing

Many web developers don't understand the importance of hashing passwords. There are many articles to explain this, but few spend the time to read and understand them. From the multitude of methods available, and the different hashing alogrithms out there, it's not a simple task to decide on which combination to use and how.

Dancer2::Plugin::Passphrase (recently ported from Dancer 1) provides a simple passwords-as-objects interface with sane defaults for hashed passwords which you can use in your web application. It uses bcrypt as the default but supports anything the Digest interface does.

Assuming we have the original user-creation form submitting a username and password:

package MyApp;
use Dancer2;
use Dancer2::Plugin::Passphrase;
post '/register' => sub {
    my $username = param('username');
    my $password = passphrase( param('password') )->generate;

    # $password is now a hashed password object
    save_user_in_db( $username, $password->rfc2307 );

    template registered => { success => 1 };
};

We can now add the POST method for verifying that username and password:

post '/login' => sub {
    my $username   = param('username');
    my $password   = param('password');
    my $saved_pass = fetch_password_from_db($username);

    if ( passphrase($password)->matches($saved_pass) ) {
        session user => $username;
        redirect param('return_url') || '/';
    }

    # let's render instead of redirect...
    template login => { error => 'Invalid username or password' };
};

Conclusion

Both Dancer2::Plugin::Auth::Tiny and Dancer2::Plugin::Passphrase are simple wrappers around basic functionality. You can check their code, it's pretty simple and straight-forward.

The strength of plugins are not necessarily in complicated and weird code, but rather in abstracting difficult things to remember and get right, and making them accessible in your web environment and integrated with your application configuration.

Authentication is not just easier with these modules, but safer.

You might also want to read up on Dancer2::Plugin::Auth::Extensible.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

The method of the setup method

Thu, 18 Dec 2014 00:00:00 -0000

The method of the setup method

Load-time setup of application variables depends on the availability of the configuration options, but we often have the requirement of being able to defer these to later. But how do we do this?

A common requirement

Many applications set up variables on load-time:

package MyApp;
use Dancer2;

my $dsn    = config->{'dsn'};
my $schema = MyApp::Schema->connect($dsn);

...

(If you're using the Dancer2::Plugin::Database, a similar thing will happen behind the scenes.

The problem we have with this is that it requires the information to be available in the configuration (accessed via the config keyword) before loading the application.

use MyApp; # config() will be called

What happens when we want to set up the variables beforehand?

I can do that, you think to yourself. Here's the code:

use Dancer2; # for config() keyword
use MyApp;

config->{'dsn'} = $new_dsn; # too late, hot plate!

Nope. Not going to work. When you call use MyApp, it will already use the config keyword to get the DSN.

Alright, so you decide to change the configuration before you load the second application. Let's try that.

use Dancer2;
config->{'dsn'} = $new_dsn;
use MyApp;

But that's not going to work, because use is compile-time, not run-time. The order is actually as following:

use Dancer2;                # happens first
config->{'dsn'} = $new_dsn; # happens last
use MyApp;                  # happens right after "use Dancer2"

You might think you could get around this with BEGIN by forcing the changes to happen before it loads the application:

BEGIN {
    use Dancer2;
    config->{'dsn'} = $new_dsn;
    use MyApp;
}

Sure. That will control the order of events by forcing the config statement to get called at compile-time, and because it's one line before the use MyApp line, it will be called before it. Oh wait...

Now you've hit a major design issue with Dancer2. We no longer play with globals because globals are evil. This means that as soon as you called use Dancer2, we created an application based on your package name.

When you change the configuration (using the config keyword), you change the configuration of the first application built. When you called use MyApp, Dancer2 had created yet another application, solely for the MyApp package. This is fully intended and is a major leading design for Dancer2.

But how do we still handle this reasonable requirement?

Moving to run-time

Instead of trying to elevate the setup code to compile-time, why not just push the application setup to run-time instead?

What if we loaded the code and imported it in runtime?

package MyApp;
use Dancer2;

my $dsn;
sub setup {
    $dsn = config->{'dsn'};
    # do something with $dsn

    get '/' => sub {...};
}

Now we just call setup method when we want to start registering routes. It's a good start. Let's make it accept parameters:

package MyApp;
use Dancer2;

my $dsn
sub setup {
    my $opts = shift;
    $dsn = $opts->{'dsn'} || config->{'dsn'};
    # do something with $dsn

    get '/' => sub {...};
}

Now we can call it from some other environment:

# mytest.t:
...

use MyApp;
MyApp->setup({ dsn => 'dbi:SQLite:dbname=:memory:' });

We can even set up our own schema object if we put that functionality in our setup method:

my $test_dsn = 'dbi:SQLite:dbname=t/corpus/test.sqlite';
my $schema   = MyApp::Schema->connect($test_dsn);
MyApp->setup({ schema => $schema });

$schema->deploy;
...

The future

Since this is such a useful bootstrapping pattern, we are still considering adding this as a built-in feature in Dancer2. While you can roll your own this easily, we could envision making this even easier.

Imagine having a bootstrap method which could be run before the routes are registered or when you actually call to_app, with optional bootstrap callbacks. It is possible and we might just blog about it in the future.

Meanwhile, Dancer applications have just enough magic to do great things while still allowing you to make use of Perl to get around load-order and encapsulation issues.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Hidden feature: Auto Pages

Wed, 17 Dec 2014 00:00:00 -0000

Hidden feature: Auto Pages

One feature we love in Dancer (both Dancer 1 and Dancer 2) is the AutoPage feature.

Auto-what?

Most websites provide web pages - multiple web pages. Many web pages will have a template and not just static content.

Rendering a template is simple:

get '/users/view' => sub {
    template 'view' => {...};
};

Assuming we have over 40 different pages, this becomes an arduous task of writing endpoints for each template, then keeping those up-to-date and continuing to add more.

If only. If only someone would have come up with a feature that allowed automatically rendering templates by path. If only someone named David Precious, who is now handling the absurd responsibility of fatherhood while leaving Sawyer X to handle the Dancer Advent Calendar all on his own, would have that idea and have already implemented it in Dancer 1, thus having it available from the early days of Dancer 2... oh wait, he did!

How

AutoPage is a simple feature that you would love. Turning it on is as simple as:

set auto_page => 1;

or in your configuration file:

auto_page: 1

Dancer will take care of the rest.

At this point, you might be wondering what it really does.

Behind the scenes

What AutoPage does is very simple: when a request was not served by a static file or by a user defined route, it looks for a template that matches the path. If it finds one, it renders it.

This means that the request /users/edit will first try to match a file, failing that it will try to match a route, and then, if still unsuccessful, it will go to AutoPage. AutoPage will search for a template under views/users/ named edit.tt, assuming your views directory is views and your templating engine default extension is .tt.

AutoPage will adhere to your views and your templating system extension, so if those change, it will still work. It will also not render the layouts themselves, so you don't need to worry about someone being a smart-ass.

Another reason it's awesome

By fully rendering a template, not just statically serving it, you have the full range of request variables. That means that any code in your templates that requires variables (or callbacks) will work:

# in layout/main.tt:
Served by [% dancer_version %]

This will work just fine. But what about variables we're adding in our code using the before_template_render hook?

# in MyApp.pm:
use Time::HiRes 'time';
hook before_template_render => sub {
    my $tokens = shift;
    $tokens->{'timestamp'} = time;
};

# in users/edit.tt:
Request timestamp: [% timestamp %]

Why would you put the timestamp there? I don't know. It's yet another contrived example that shows how you can add variables that will be accessible to the template rendered by AutoPage.

Conclusion

The AutoPage feature is probably one of the nicest subtle features in Dancer. We don't hide it, but we also don't make it abundantly clear to users how awesome it is and how you most likely want to use it.

At the end of the day, this feature also serves as yet another reason to buy David Precious a drink, which will finally force him to come to some public Perl event. This is all part of our secret plan.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Command line dancing

Tue, 16 Dec 2014 00:00:00 -0000

Command line dancing

Some of the changes Dancer2 has introduced were the removal of all import tags:

use Dancer ':syntax'; # use Dancer2;
use Dancer ':script'; # use Dancer2;
use Dancer ':tests';  # use Dancer2;

Now you simply call use Dancer2 for all of these. It will create an application and provide keywords. If you want to create a PSGI application from it, simply call to_app on your class. Otherwise, it doesn't get in the way.

But I need it

While the last paragraph actually summed up this entire article, it probably wasn't very clear.

A useful feature of going through Dancer itself is that it will read the configuration files, parse them, understand them, and even create objects lazily using them.

People often write scripts that have to then duplicate what Dancer does. It's quite understandable that people wish to be able to use Dancer in their scripts to have access to the configuration and the generated objects.

Reaching the config

By importing Dancer2 in your command line scripts, you have full access to the configuration using the imported keywords:

# myscript.pl:
use Dancer2; # strict && warnings included

# this works:
my $appname = config->{'appname'};

# and so does this:
my $appname = app->name;

You shouldn't worry about myscript.pl becoming an app and starting a web server. Yes, as soon as it hits use Dancer2 it will create a Dancer2 application behind the scenes, and that app can be run at any given point in time by calling dance or the preferred method to_app on it, but unless you actually call those, this app will simply be created in the background.

You will still be able to access all the configuration from it without having to run it under a web environment.

Small example

One small example of wanting to access the configurations from a command line script would be to run some sort of import to or export from a database.

Being able to access the configuration file and having the classes that are automatically inflated from that configuration file is mighty useful.

# myscript.pl:
use Dancer2
use Dancer2::Plugin::Database;

my $user_id = $ARGV[0] or die "$0 <USER ID>\n";
$user_id =~ /^[0-9]+$/ or die "User ID must be a positive number\n";

my $sth = database->prepare( 'select * from users where id = ?' );
$sth->execute($user_id);

...

Conclusion

If you want to create command line applications that use the Dancer2 structures, you can simply load Dancer2 and use the keywords it provides you.

What does this give you access to?

Dancer keywords
config probably being the most helpful one, but template will also be available just like other engine-related keywords are - explained below under Engines.
Keywords created by plugins
This means you can make use of any functionality they provide outside of a request.
Engines
This means you can use the facilities your web application has when logging events (using debug, warning, and error), accessing or setting session information, serializing, and even rendering templates (using the template keyword).

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Outreach Program for Women (OPW) and Dancer

Mon, 15 Dec 2014 00:00:00 -0000

Outreach Program for Women (OPW) and Dancer

The Outreach Program for Women (OPW) helps women work on Open Source and Free Software applications while receiving a salary and having a mentor by their side.

The Perl Foundation (TPF) is participating this year again, and this year we have the pleasure of participating with Dancer!

Our participant

Our applicant, Snigdha Dagar, has been accepted to work on Dancer2's documentation overhaul!

She had already contributed the first draft of the Dancer2::Manual::Migration document to help users with upgrading from Dancer to Dancer2.

The documentation overhaul

The documentation overhaul was deemed a crucial component in making the upgrading from Dancer 1 to Dancer 2 more attainable.

The current documentation has been copied over from Dancer 1, but is not completely compatible. Some things have changed and many features have been added. Unfortunately in the frenzy of continuing upgrading Dancer and its abilities, along with important improvements, we lack the tuits to resolve the outstanding problems with the current documentation.

Starting from scratch would be a tremendous amount of work, and whether it will be less or more work than going through the current documentation with a fine comb and sorting it out still does not resolve the problem with the lack of personnel for this task.

What we need is human resources - and not the department. :)

There is a plan

Snigdha has provided a plan for overhauling the documentation, which includes researching other FOSS projects, provide introductory material, How-to guides, and cohesiveness of the overall documentation.

Her work will help resolve the documentation problems, assure consistency, accuracy, and above all, a well structured, helpful documentation for such an important project.

There will be updates

We are excited and delighted to have her help in these upcoming months and we will be providing updates on the progress made.

We are certain Snigdha would appreciate any guidance and assistance from the community. Feel free to suggest corrections, improvements, what to focus on, what to not focus on, and how to assure the success of her work and the project.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Writing plugins for Dancer2

Sun, 14 Dec 2014 00:00:00 -0000

Writing plugins for Dancer2

Plugins provide you with additional keywords to decouple predefined behavior to make it easier on you so you won't need to implement the same thing over and over again.

Let me show you how simple it is to write a plugin of your own.

(This explains the current API to plugins. Refer to Dancer2::Plugin for up-to-date documentation.)

A plugin that does nothing (so far)

All you need to do is use Dancer2::Plugin and that will provide you with all the keywords you need to write the plugin.

package Dancer2::Plugin::Kitteh;
use Dancer2::Plugin;

# we do nothing, just like most cats do

register_plugin;

1;

Introducing keywords

You can introduce new keywords the application will receive when it uses your plugin using the register keyword:

register meow => sub {
    my ( $dsl ) = plugin_args(@_);
    my $app = $dsl->app;
};

This is how we introduce the meow keyword to the user.

We use plugin_args to make sure we get the plugin arguments. This is for compatibility.

The keyword receives an object which represents the DSL object the app is connected to. You can use it in order to access the Dancer2 core application connected to the user's scope.

We can also control whether a keyword is app-global, meaning it can be called from anywhere in an app or only from a route, which means during a request:

register meow => sub {
    debug 'Meow!';
}, { is_global => 0 };

Route decorators

Some plugins generate routes from other routes, which makes them look a little bit like route decorators. Take Dancer2::Plugin::Auth::Tiny for example:

get '/private' => needs login => sub { ... };

The way it works is by taking the route sub as a parameter and creating its own route which calls it. We can do the same.

package Dancer2::Plugin::OnTuesday;
# ABSTRACT: Make sure a route only works on Tuesday
use Dancer2::Plugin;

register on_tuesday => sub {
    my ( $dsl, $route_sub, @args ) = plugin_args(@_);

    my $day = (localtime)[6];
    $day == 2 or return pass;

    return $route_sub->( $dsl, @args );
};

register_plugin;

Now we can use this plugin as such:

package MyApp;
use Dancer2;
use Dancer2::Plugin::OnTuesday;

get '/' => on_tuesday => sub { ... };

# every other day
get '/' => sub { ... };

Reading the configuration

While a user can change the configuration using both the configuration file and the set keyword, we need a single source for all configuration options for our plugin. This is handled automatically using the plugin_setting keyword:

register meow => sub {
    my $dsl = shift;
    my $vol = plugin_setting->{'volume'} || 3;
};

Extra tricks

There are a few additional tricks available which weren't covered here, such as running code on import, registering additional hooks, and executing hooks. They are all documented in Dancer2::Plugin.

Conclusion

The plugin infrastructure in the upgrade from Dancer 1 to Dancer 2 has been problematic. We've tried various methods to allow plugins to coexist, then to specify how they work, and finally to provide a proper shim layer to enable working with both smealessly.

Unfortunately all of these methods had problems and we resulted in separating the plugins. This does not mean that writing a plugin for Dancer 2 is difficult. In fact it's fairly simple.

We're working on making them even more comfortable and remove any lingering problems, but as you can see they are already easy to write.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

The test core: How we test in Dancer2 core

Sat, 13 Dec 2014 00:00:00 -0000

The test core: How we test in Dancer2 core

The Dancer2 core test suite is improving significantly. We have made vast changes in how we sort our tests and we now use Plack::Test to write tests.

If you're interested in contributing, we would not only praise you for years to come, but also shower you and your family with gifts. But before you can contribute tests, it's important you understand our practices and conventions when writing tests.

Basics

There are a few guidelines for the tests which we try to maintain when writing tests:

use_ok for testing classes load
Declare the number of tests
Use subtests or code blocks for scoping
Try to reach as much coverage as possible
Reduce convenience modules and don't make them mandatory
Thus, skip if those modules aren't available
Provide each test with a test name - no empty tests

Testing classes

The main tests are actually for classes. As explained, we store these tests in t/classes/, under a directory for each class we test.

Let's take a look at the Dancer2::Core::Factory object tests. Since it simply creates engine objects and contains only a single method, the test is rather simple.

Boilerplate:

use strict;
use warnings;
use Test::More tests => 5;

As explained, this is not a fast and hard rule, but we prefer to count the number of tests we have, to make sure the test doesn't miss a test someone has accidentally put on a condition.

We test loading:

use_ok('Dancer2::Core::Factory');

Then we test the object itself:

my $factory = Dancer2::Core::Factory->new;
isa_ok( $factory, 'Dancer2::Core::Factory' );
can_ok( $factory, 'create' );

You might notice we pick descriptive names. We try to avoid variables names $f in our core code as well as in our tests.

Now we can test the create method, which is actually a class method, so holding the factory object is not required. We generally avoid class methods, but in this class we're a bit more lax about it:

my $template = Dancer2::Core::Factory->create(
    'template', 'template_toolkit', layout => 'mylayout'
);

isa_ok( $template, 'Dancer2::Template::TemplateToolkit' );
is( $template->{'layout'}, 'mylayout', 'Correct layout set in the template' );

That's it. Pretty simple.

Testing DSL

In order to test DSL keywords, we need to have a valid application who uses Dancer2 and then try its DSL keywords. That's actually simple enough.

The test t/dsl/app.t provides our test for the app keyword, which returns the current Dancer2::Core::App a package is using.

First we have our boilerplate:

use strict;
use warnings;
use Test::More tests => 2;
use Plack::Test;
use HTTP::Request::Common;

Now we define an application in the code in its own scope so it doesn't interfere with the test itself, but be available to it:

{
    package App;
    use Dancer2;
    get '/' => sub {
        my $app = app;
        ::isa_ok( $app, 'Dancer2::Core::App' );
        ::is( $app->name, 'App', 'Correct app name' );
    };
}

This application uses Dancer2 and thus receives the keywords. Since our test is written in the implicit main package, the double colon shorthand for main:: class resolution allows it to call ::isa_ok to reach the test's isa_ok function (provided by Test::More) and ::is allows it to reach the test's is function (also provided by Test::More).

This test checks that if you reach the main path, it will call the app keyword and check it received an instance, that it is the right instance, and that the name was even set correctly.

We still need to write the code to reach it:

Plack::Test->create( App->to_app )->request( GET '/' );

There.

Often we return values from a web request, but in this specific case we're really interested in an object which the route receives and doing it by sending it back as a response would just be overengineering.

What about external files?

Some tests might require some additional files to work. Perhaps you're checking the side-effect of reading from the configuration file or a piece of code that works with the file system.

One example is t/issues/gh-639 which has additional directories, each have their own config.yml and test file to check against different configuration files as part of a general issue opened.

Conclusions

Following the guidelines you will be able to help us improve our tests, move our old ones, and provide new tests for issues you or others might open.

We're very interested in having a good test coverage and providing reliable behavior. Tests are a major component of it and we would always be more than happy to receive more help in that field.

Next time you open an issue, consider writing a test for it. If you're not sure how, please contact us. We would love to help. :)

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

A test to remember: testing your web application

Fri, 12 Dec 2014 00:00:00 -0000

A test to remember: testing your web application

Perl has an extensive culture of testing which originated in the Perl language itself, carrying a staggering number of tests for its own features.

We write a multitude of tests for every single bit in our distributions and we test them across different platforms, setups, versions of modules and applications, and even on different versions of perl[1] itself.

For non-Perl regulars, you might think I'm exaggerating. Oh no, I'm not. Check out CPANTS, our CPAN Testing Service for more details on our amazing free Perl testing service.

It's safe to say: we love tests.

[1] We use Perl (uppercase) for the language and perl (lowercase) for the interpreter.

But in the land of web...

In the world of web, testing is trickier. A web application runs on a web server. That is usually an application written in another language (often C or C++), a long-running process with lots of configuration files and behavior that is hard to decouple.

LWP Gave us LWP::UserAgent which gave us WWW::Mechanize. As long as we had a web server running, we could just write some tests (turned on with an environment variable) that will make requests and test the results, producing proper TAP. There's even Test::WWW::Mechanize to make it smoother.

However, these don't handle the problem with testing in a web environment: the web server itself.

Enter PSGI

Any web application built on a PSGI framework allows to approach the problem from an entirely different perspective.

PSGI applications (such as those created with Dancer) are actually code references fed to a web server. This means that as long as you create the proper request (specified by the PSGI docs), you can call that application code reference with it and receive a response. No server is actually necessary. The server is merely there to create a proper request.

Plack, the set of utilities for PSGI applications, comes with a special module to help this process: Plack::Test.

What Plack::Test does is truly fantastic: it receives a common web request (using standard HTTP::Request objects), fakes a web server in order to create a proper PSGI request, and sends it to the web application. When the web application returns a PSGI response (which Dancer applications do), it will then convert it to a common web response (as a standard HTTP::Response object).

This allows you to then create requests in your test, create the code reference for your web application, call them, and receive a response object, which you can then test to your heart's delight.

Let's take a look at a few examples.

Basic example

Assuming we have a web application:

# MyApp.pm
package MyApp;
use Dancer2;
get '/' => sub {'OK'};
1;

Now we want to write a test for it. Let's create base.t:

# base.t
use strict;
use warnings;
use Test::More tests => 2;
use Plack::Test;
use HTTP::Request;
use MyApp;

Now let's create a coderef for our application using the to_app keyword:

my $app = MyApp->to_app;

That was easy. Now let's create a test object from Plack::Test for our application:

my $test = Plack::Test->create($app);

Now we can call requests on it. We'll create our first request object and send it to our test object to receive a response:

my $request  = HTTP::Request->new( GET => '/' );
my $response = $test->request($request);

We can now test it:

ok( $response->is_success, '[GET /] Successful request' );
is( $response->content, 'OK', '[GET /] Correct content' );

Putting it all together

If we put all our code together and remove some excess, we can come up with the following test file:

# base.t
use strict;
use warnings;
use Test::More;
use Plack::Test;
use HTTP::Request::Common;
use MyApp;

my $test     = Plack::Test->create( MyApp->to_app );
my $response = $test->request( GET '/' );

ok( $response->is_success, '[GET /] Successful request' );
is( $response->content, 'OK', '[GET /] Correct content' );

done_testing();

It might seem like too much boilerplate, but there's enough modules to help you reduce that.

Subtests

We also separate our tests using Test::More's subtest functionality, thus creating multiple self-contained tests that don't overwrite each other.

Assuming we have a different app that has two states we want to test.

# MyApp.pm
package MyApp;
use Dancer2;
set serializer => 'JSON';

get '/' => sub {
    my $user = param('user');

    $user and return { user => $user };

    return {};
};

1;

This is a very contrived example of a route that checks for a user parameter. If it exists, it returns it in a hash with the key 'user'. If not, it returns an empty hash. Useful? Probably not, but a good example for having two tests.

# param.t
use strict;
use warnings;
use Test::More;
use Plack::Test;
use HTTP::Request::Common;
use MyApp;

my $test = Plack::Test->create( MyApp->to_app );

subtest 'A empty request' => sub {
    my $res = $test->request( GET '/' );
    ok( $res->is_success, 'Successful request' );
    is( $res->content '{}', 'Empty response back' );
};

subtest 'Request with user' => sub {
    my $res = $test->request( GET '/?user=sawyer_x' );
    ok( $res->is_success, 'Successful request' );
    is( $res->content '{"user":"sawyer_x"}', 'Empty response back' );
};

done_testing();

We could, of course, use JSON to decode and check for a proper object, which would be the right thing for a bigger response. Did I say contrived yet? :)

Cookies

One interesting requirement is being able to handle cookies, mostly used for maintaining sessions. We can use Test::WWW::Mechanize::PSGI or LWP::Protocol::PSGI, I personally prefer using HTTP::Cookies directly.

Taking the previous test, assuming it actually creates and uses cookies for sessions, all we would need is the following:

# ... all the use statements
use HTTP::Cookies;

my $jar  = HTTP::Cookies->new;
my $test = Plack::Test->create( MyApp->to_app );
my $host = 'http://127.0.0.1';

subtest 'A empty request' => sub {
    my $res = $test->request( GET "$host/" );
    ok( $res->is_success, 'Successful request' );
    is( $res->content '{}', 'Empty response back' );
    $jar->extract_cookies($res);
    ok( $jar->as_string, 'We have cookies!' );
};

subtest 'Request with user' => sub {
    my $req = GET "$host/?user=sawyer_x";
    $jar->add_cookie_header($req);
    my $res = $test->request($req);
    ok( $res->is_success, 'Successful request' );
    is( $res->content '{"user":"sawyer_x"}', 'Empty response back' );
    $jar->extract_cookies($res);

    ok( ! $jar->as_string, 'All cookies deleted' );
};

done_testing();

Here we create a cookie jar, make sure all our requests and responses work with it, and we can even check for existing cookies, as well as cookies that were deleted by the response, since HTTP::Cookies will understand a delete request (by setting an older time) and delete it.

You will notice that we use a full domain. This is because HTTP::Cookies needs the entire URL (including the scheme part) to make the cookies work.

Accessing the configuration file

By importing Dancer2 in your command line scripts, you have full access to the configuration using the imported keywords:

use strict;
use warnings;
use Test::More;
use Plack::Test;
use HTTP::Request::Common;
use MyApp;
use Dancer2;

my $appname = config->{'appname'};
diag "Testing $appname";

# ...

In this example we use the config keyword to access the configuration. That's all you have to do if you want to access it as part of your test.

Testing environment

You can also set your own environment file for testing, such as environments/testing.yml and then load it in your test script:

BEGIN { $ENV{'DANCER_ENV'} = 'testing' }
# ...

Now you can insert your own configurations.

Setup method

My current favorite method for dynamically handling onfigurations in my Dancer2 applications is by having a setup method. When loading my applications in my handlers, I call that method, and it then checks the configuration for variables.

This allows me to tweak and change the configuration before calling the setup method, thus letting me bootstrap my own changes, such as in testing scripts.

A separate article about this trick just might appear on the Advent Calendar this year.

Where are the test helpers?

Both Dancer 1 and 2 have Dancer::Test and Dancer2::Test, respectively, but in Dancer2 it is currently not recommended.

The reason is that, because they weren't written with PSGI in mind, they are bending over backwards in order to fake the request, run it against a fake dispatcher, and then return a fake response.

At the end of the day, it isn't only buggy and took a lot of our time and efforts into maintaining it, but it is also counter-intuitive.

We might reinstate it later, once we've redesigned it to simply be a helper layer on top of Plack::Test.

For now, we suggest to avoid it and go directly for Plack::Test.

Conclusion

Writing tests is a lot of fun and they help us make sure our applications work the way we want them to work. They help flesh out features, APIs, and maintain their correctness throughout changes, big and small.

Writing tests for web application used to be difficult, but now, with the magic of PSGI, it is simple and straight-forward.

You can read more about testing in Dancer2::Manual::Testing.

Our next article will cover how we write tests ourselves in the Dancer2 core.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Serial Serializer - making API writing easier

Thu, 11 Dec 2014 00:00:00 -0000

Serial Serializer - making API writing easier

One of Dancer's engines is the serializer, which helps you with writing APIs.

Serializers essentially do two things:

Deserialize incoming requests
When a user makes a request with serialized input, the serializer automatically deserializes it into actual input parameters.
Serialize outgoing responses
When you return a data structure from a route, it will automatically serialize it for you before returning it to the user.

Configuring

In order to configure a serializer, you just need to pick which format you want for encoding/decoding (from the available Serializer namespace - Dancer::Serializer for Dancer 1 and Dancer2::Serializer for Dancer2) and set it up using the serializer configuration keyword.

It's recommended to explicitly add it in the actual code instead of the configuration file so it doesn't apply automatically to every app that reads the configuration file (unless that's what you want):

package MyApp;
use Dancer2;
set serializer => 'JSON'; # Dancer2::Serializer::JSON

...

Using

Now that we have a serializer set up, we can just return data structures:

get '/' => sub {
    return { resources => \%resources };
};

When we return this data structure, it will automatically be serialized into JSON. No other code is necessary.

We also now receive requests in JSON:

post '/:entity/:id' => sub {
    my $entity = param('entity');
    my $id     = param('id');

    # input which was sent serialized
    my $user = param('user');

    ...
};

We can now make a serialized request:

$ curl -X POST http://ourdomain/person/16 -d '{"user":"sawyer_x"}'

It's all automatic.

Always or never

In previous versions of Dancer2 (and in Dancer 1), serializers were only used when the data is a reference.

There were some issues with that:

Unclear behavior
You could have mixed routes that return serialized and non-serialized content, which is a source for confusion and inconsistency.

Now you simply handle this by providing a package that has a serializer and a package that renders content. You can then separate them to different mounting points using Plack::Builder as explained in a previous article and showed below.
What if the data is a string?
Some serializers actually handle strings. Dancer2::Serializer::CBOR, for example, can serialize strings. This previous ref-only behavior would render those serializers broken - not good.

Instead, now everything is sent to serializers. Bad requests receive warnings and errors.

Hooks

While we're still working on adding an error hook, you can already hook into the serializer with a before and after:

hook before_serializer => sub {...};
hook after_serializer  => sub {...};

App-specific feature

Remember that serializers are engines. They affect a Dancer Application, which means that once you've set a serializer, all routes within that package will be serialized and deserialized. This is how the feature works.

As suggested above, if you would like to have both, you need to create another application which will not be serialized.

A common usage for this is an API providing serialized endpoints (and receiving serialized requests) and providing rendered pages.

# MyApp.pm
package MyApp;
use Dancer2;

# another useful feature:
set auto_page => 1;

get '/' => sub { template 'index' => {...} };

# MyApp/API.pm
package MyApp::API;
use Dancer2;
set serializer => 'JSON'; # or any other serializer

get '/' => sub { +{ resources => \%resources, ... } };

# user-specific routes, for example
prefix => '/users' => sub {
    get '/view'     => sub {...};
    get '/view/:id' => sub {...};
    put '/add'      => sub {...}; # automatically deserialized params
};

...

Then those will be mounted together for a single app:

# handler: app.pl:
use MyApp;
use MyApp::API;
use Plack::Builder;

builder {
    mount '/'    => MyApp->to_app;
    mount '/api' => MyApp::API->to_app;
};

There ya go!

Conclusion

Serializers make it much easier to write API interfaces. We recommend using them in order to save yourself on typing. However, use caution knowing they will handle all input and all output and affect all routes in a package.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

What's in an appname?

Wed, 10 Dec 2014 00:00:00 -0000

What's in an appname?

New feature in Dancer2: appname

One change we absolutely love is the appname import option, introduced in Dancer2 0.150000.

What's a Dancer App again?

It's worth repeating the explanation of what a Dancer2 application is, since this might confuse a developer with a PSGI application.

A Dancer2 App is simply a package containing routes and engines which receives a request and returns a response. The engines are not mandatory, but might be a template, a session, a logger, and a serializer.

A PSGI application is a code reference which receives a request as a hash of environment variables (from a server using a Plack handler) and returns a proper PSGI response which could be an array reference or a code reference.

Too many apps!

As soon as you import Dancer2 (by calling use Dancer2), you create a Dancer2 App. It will use your class name (defined by either the package function in Perl or the default in Perl: main) to define the Dancer2 App name. This is how Dancer2 will recognize your application.

This introduces an interesting situation. If you wish to separate your App to multiple files, you're actually creating multiple applications.

So what?

This means that any engine defined in an application, because the application is a complete separate scope, will not be available to a different application:

package MyApp::User {
    use Dancer2;
    set serializer => 'JSON';
    get '/view' => sub {...};
}

package MyApp::User::Edit {
    use Dancer2;
    get '/edit' => sub {...};
}

These are two different Dancer2 Apps. They have different scopes, contexts, and thus different engines. While MyApp::User has a serializer (the JSON one) defined, MyApp::User::Edit will not have that configuration.

If you want the path /edit to have a serializer also, you will need to define one for MyApp::User::Edit. This is not just tedious, but when you mix this with the to_app method (described in the previous article, you have a problem:

use MyApp::User;
use MyApp::User::Edit;

use Plack::Builder;

builder {
    mount '/' => MyApp::User->to_app;
    # what about MyApp::User::Edit ???
};

If / is taken by MyApp::User, where do we put MyApp::User::Edit?

We don't actually wish MyApp::User::Edit to be another application, we just want it to be a comfortable extension to the regular app, providing additional routes that are part of the same already-existing application.

While you might think loading MyApp::User::Edit within MyApp::User will help to just extend it, that is not the case. It will still fail.

appname to the rescue!

appname saves the day

By using the import option appname, we can ask Dancer2 to extend an App without creating a new one:

package MyApp::User {
    use Dancer2;
    set serializer => 'JSON';
    get '/view' => sub {...};
}

package MyApp::User::Edit {
    use Dancer2 appname => 'MyApp::User'; # extending MyApp::User
    get '/edit' => sub {...};
}

Now, by using appname when importing, we can ask this App to simply extend an existing one (or one will be created if it does not exist yet) and add the routing there.

The serializer will be available, it will have the same configuration, and it will even represent all of the routes with the same internal application object.

# app.pl
use MyApp::User;
use MyApp::User::Edit;

# single application composed of routes provided in multiple files
MyApp::User->to_app;

Usually we would load classes that contain additional routes inside the original class they extend:

package MyApp::User {
    use Dancer2;
    use MyApp::User::Edit;
    set serializer => 'JSON';
    ...
}

This way we only need to load one when we create an app:

# app.pl:
use MyApp::User;
MyApp::User->to_app;

Conclusion

The import option appname allows you to seamlessly extend Dancer2 Apps without creating unnecessary additional applications or repeat any definitions.

This allows you to spread your application routes across multiple files and allow ease of mind when developing it, and accommodate multiple developers working on the same codebase.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

New feature in Dancer2: to_app

Tue, 9 Dec 2014 00:00:00 -0000

New feature in Dancer2: to_app

Dancer2 0.151000 has made a major change to the dispatching mechanism and provided a new method: to_app. What was the change, why was this keyword introduced, and why should we use it now?

The way things were

The original dispatching mechanism in Dancer2 would take all of your Dancer2 applications and wrap them with a loop that tries to match their routes. Once you know that a Dancer App is basically a unit of routes with its own set of engines (such as a template, a session storage, a logger, and a serializer - not all required), you might begin to see a problem here.

If you have two applications (meaning two packages that have use Dancer2 in them) called MyApp::A and MyApp::B, and you call the dance keyword to create a full fledged PSGI application from them, Dancer will try to match a request against each one. It will start with MyApp::A and then MyApp::B.

"Is it always in this order?", you might ask. Actually, it isn't. Since it is internally stored in an array (which is at least deterministic, right?), it is actually checked in the order in which you loaded them into memory.

If your handler is:

use MyApp::B;
use MyApp::A;
use Dancer2;

dance;

It will now check MyApp::B first and MyApp::A last.

In fact, it will check MyApp::B first, MyApp::A second, and the last will be whatever package this snippet of code you saw is in - in case it has any routes configured in it before dance was called.

Why is this bad?

So you might be wondering, "Okay, there's an order issue here, but why should I care? Eventually it reaches my route and works fine."

You are absolutely right - in most cases. However, if both applications have the same route defined, you won't necessarily know which route was actually hit.

If you're preloading applications into memory (which is a good idea in large infrastructures), your resulting application might hit someone else's route instead.

If you're using the AutoPage feature, you might snag on a different app's template. If one app has a Serializer defined, you might return serialized output. If different apps have a different Session engine, that's even worse. That's tantamount to a security risk - although we try to minimize that risk internally by moving session objects around so you don't hit a mistaken session.

What can we do?

The first change we made, back in Dancer2 0.150000, was to call the psgi_app keyword with specific application names, either by class name or by a regular expression. This allowed you to create PSGI applications from specific Dancer Apps:

# preload everything
use MyApp::A;
use MyApp::B;
use MyApp::C;
use MyApp::D;
use MyApp::E;

# an app for A, B, C
my $abc_app = Dancer2->psgi_app([ qr{^MyApp::[ABC]$} ]);

# an app for D, E
my $d_app = Dancer2->psgi_app([ 'MyApp::D', 'MyApp::E' ]);

This allows us to load into memory all the applications we have and then create a PSGI application from only some of them.

Here is another example which also joins multiple apps into a single, bigger PSGI app with Plack::Builder:

use MyApp::A;
use MyApp::B;
use MyApp::C;

use Plack::Builder;

builder {
    mount '/'        => Dancer2->psgi_app(['MyApp::A']);
    mount '/special' => Dancer2->psgi_app([ qr{^MyApp::[BC]$} ]);
};

This was a major improvement, but it still didn't address the core problem.

Fine. I'll bite. What was the core problem?

I'm glad you asked!

The code problem was that the dispatching was the duty of the core dispatcher object (an instance of Dancer2::Core::Dispatcher) instead of the app object itself (Dancer2::Core::App). Since the routing should have been done in the app object (the lowest context for engines and routes), it made sense to move it there instead of the dispatcher object.

We've done so in Dancer2 0.151000, a version later.

While retaining the old behavior of the psgi_app keyword, we introduced one major addition: the to_app class method.

to_app or not to_app

While perhaps not the same question Shakespeare asked, it is a noble one just the same.

The to_app method creates a PSGI app from a single application - not from all of your applications or a PSGI app that is restricted to specific ones.

This means you can now treat a Dancer2 App as its own unit that can be composed into any path you like.

The following:

MyApp->to_app;

gives the same behavior to the user as:

Dancer2->psgi_app(['MyApp']);

except it does it using quite a few workarounds.

Instead of using the awkward Dancer2->psgi_app(), which provides the same behavior as dance(), you could use to_app. This won't require calling use Dancer2 in your handler, since it's a method on your Dancer2 application class.

When you have multiple Dancer2 applications, you can compose them together using either Plack::App::URLMap or the wrapper syntax for it available in Plack::Builder:

use MyApp::Main;
use MyApp::Admin;

builder {
    mount '/'      => MyApp::Main->to_app;
    mount '/admin' => MyApp::Admin->to_app;
};

The effect of the mounting is that these applications will be completely separate and Plack::Builder will assure only the appropriate application handles a given request.

Instead of creating a loop around all of them, you provide each app with its own dispatching mechanism, in charge only for the routes defined inside it.

Another effect of mounting is that the mounting point (/admin, for example) is stripped from the path the application needs to match. This means that MyApp::Admin doesn't need to define /admin/view, but just /view.

This has become such a useful and important feature that if you try to call psgi_app on an application (instead of on Dancer2 or Dancer2->runner, it will effectively call to_app underneath.

Conclusion

In short, yes. We do want to_app. :)

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Sweetening the test suite

Mon, 8 Dec 2014 00:00:00 -0000

Sweetening the test suite

The testing suite of a project is an important component, allowing users and developers to maintain trust in the code. Having an organized testing suite makes it easier for developers to add new tests, improve existing tests, and make sure tests are evenly covering the variety of code from all of its angles, as well as making it easier for contributors to help out.

The sweet suite

While mayhem ruled the testing suite of Dancer (both version 1 and 2), we've recently started rearranging it in order to handle the various situations our code works under.

Here's the new structure:

t/classes/

The main component of the testing suite are the classes. In Dancer2 all the DSL implementation is built on top of core classes. This allows us to decouple the keywords the user uses and their implementation.

Under the t/classes/ we run tests for classes only. This should not run web requests, understand the web environment, or anything of the sort. Some exceptions may apply, but we try to minimize those.

Each class has a directory for itself, allowing you to quick find it, and add tests to it. Each method gets its own test file. We test roles as well, of course. In each directory we can provide static files fo test various scenarios.

t/dsl/

The user-visible interface of the Dancer web framework is the DSL, which are those keywords we use to define routing.

The DSL is tested under the t/dsl/ directory. Inside it each DSL keyword has its own file. Keywords requiring more than a single test, or requiring static files (such as sample configuration files), receive their own directory.

We're still porting the old DSL keywords into clean tests inside the t/dsl/ directory, which means this directory is still relatively small and most of the DSL tests are strewn across the general t/ directory.

We wouldn't say no to any help offered. :)

t/issues/

Regression tests offer a test which isn't covered by the above described categories. We use regression tests to check specific situations, rather than code paths.

Since these situations arise in tickets (and tickets can always be opened for them if not), we created a directory holding all of these tests, sorted by the tracker and the ticket ID in that tracker.

Often times a good way to handle an issue is to provide a test, which can then allow verifying it was fixed.

For example, we have t/issues/gh-723.t to make sure that scenario doesn't pop up again as a bug.

Conclusion

Nowadays when we want to play with tests, we already know where they go and how to write them. Now you do too, which makes it possible for you to write up some tests when providing a fix or a feature, or when you just want to improve the testing coverage.

Consistency in the test suite is absolutely vital in order to continually improve the testing coverage and make sense of the testing code.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Doing an API Mashup with Dancer

Sun, 7 Dec 2014 00:00:00 -0000

Doing an API Mashup with Dancer

I was delighted to receive a Twitter message from Su-Shee on an article about mashing up an API call with Sinatra:

.@PerlSawyer tiny total beginner's #sinatra example, should be blatantly copied for #dancer... https://t.co/SlSia0f92a "learn how to API" :)
— Su-Shee (@sheeshee) December 6, 2014

You might wonder how a version of it would look like in Dancer. Shouldn't be a problem. Let's have a go at it!

Prior work

If you haven't had the time or inclination to read the aforementioned article (which we actually recommend), here is a brief description of our task.

We want to create an app that makes a request to a weather API and then displays it dynamically in a web page.

To everything a start

Setting up the application was relatively quick. Open up your favorite editor on a new file. Let's call it mashup.pl:

Other than Dancer2 for defining routes, we will use HTTP::Tiny to make the weather API request, JSON to decode it from JSON format, and finally File::Spec to provide a fully-qualified path to our template engine. That last bit will be explained later on.

use JSON;
use Dancer2;
use HTTP::Tiny;
use File::Spec;

Configuration

Now we set up some variables for our web application. We will use the excellent Template::Toolkit template system. It is simple and easy to work with.

Dancer searches for our templates in our views directory, which defaults to views directory in our current directory. Since we want to put our template in our current directory, we will configure that. However, Template::Toolkit does not want us to provide a relative path without configuring it to allow it. This is a security issue. So, we're using File::Spec to create a full path to where we are.

We also unset the default layout, so Dancer won't try to wrap our template with another one. This is a feature in Dancer to allow you to wrap your templates with a layout when your templating system doesn't support it. Since we're not using a layout here, we don't need it.

set template => 'template_toolkit';       # set template engine
set layout   => undef;                    # disable layout
set views    => File::Spec->rel2abs('.'); # full path to views

A lot of explanation and only three lines of code. :)

Now, we define our URL:

my $url = 'http://api.openweathermap.org/data/2.5/weather?id=5110629&units=imperial';

Route - the gist of it

The main component is our web route. We will define a main route which, upon a request, will fetch the information from the weather API, decode it, and then display it to the user.

Route definition:

get '/' => sub {
    ...
};

Editing the stub of route dispatching code, we start by making the request and decoding it:

# fetch data
my $res = HTTP::Tiny->new->get($url);

# decode request
my $data = decode_json $res->{'content'};

The data is not just a flat hash. It's a deep structure. To make this post simpler, we will filter it for only the simple keys in the retrieved data:

my $metrics = { map +(
    ref $data->{$_} ? () : ( $_ => $data->{$_} )
), keys %{$data} };

All that is left now is to render it:

template index => { metrics => $metrics };

Oh, and to make this into a runnable-application, we add the magic word:

dance;

And the template

We can't forget about the HTML code of course. The following is our index.tt template file:

<table>
  <th>The Weather in Buffalo, NY, USA</th>
  <tr>
    <td>
    [% FOREACH metric IN metrics.keys %]
        <tr><td>[% metric %]</td><td>[% metrics.$metric %]</td></tr>
    [% END %]
    </td>
  </tr>
</table>

<hr noshade>

Powered by the <a href = 'http://openweathermap.org/' target = _new>Open W
eather Map API</a>

Et Voila!

All together now

Here is our complete app:

use JSON;
use Dancer2;
use HTTP::Tiny;
use File::Spec;

set template => 'template_toolkit';
set layout   => undef;
set views    => File::Spec->rel2abs('.');

my $url = 'http://api.openweathermap.org/data/2.5/weather?id=5110629&units=imperial';
get '/' => sub {
    my $res     = HTTP::Tiny->new->get($url);
    my $data    = decode_json $res->{'content'};
    my $metrics = { map +(
        ref $data->{$_} ? () : ( $_ => $data->{$_} )
    ), keys %{$data} };

    template index => { metrics => $metrics };
};

dance;

And the result we have at the end:

The Weather in Buffalo, NY, USA

id	5110629
cod	200
name	Buffalo
base	cmc stations
dt	1417885200

Conclusion

Many of the odder configuration options were actually due to writing an application outside the defaults of a normal production-ready application.

If we had a views directory, we wouldn't need File::Spec and to configure the views. If we had used a layout, which we would normally use, we wouldn't need to disable the layout using the layout option. If we tried to account for all data returned, we would also write a template that supports a deeper structure, or dumped it to HTML from its hash representation, which would probably had been easier.

At the end, the production code would share all of these comments and end up both smaller and simpler. Of course, our production code would check for errors on fetching the data and decoding it, which we have conveniently ignored. :)

One clear difference between our implementation and the Sinatra one is we don't actually stick any HTML in the code itself. There is no need. We simply throw it to the templating engine.

With light web frameworks writing your application is usually as straight-forward and simple as it seems in our head.

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Static noise - Static file serving in Dancer2

Sat, 6 Dec 2014 00:00:00 -0000

Static noise - Static file serving in Dancer2

All too often beginners would stumble over the behavior of static file serving in Dancer2. The idea made sense on paper but somehow, when you put it in practical terms, it wasn't just a hurdle but a real pain to constantly headbutt without realizing, "damn it, that again!"

The good news is we fixed it. The bad news is you will have to read this article to understand what we're talking about. The treat is we will explain how it was fixed after we explain what was wrong.

Ex-static

The logic for request serving in Dancer 1 is simple. It consists of the following steps run in order:

Try to render a static file from public/
Try to render an action - a user's route
Try to render an AutoPage - a user's template
Give up and render a 404 page

In Dancer2 these items were broken down into composable units known as handlers. Dancer2::Handler::File serves static files (which the DSL keyword send_file uses internally) and Dancer2::Handler::AutoPage renders template files.

These handlers were, when configured to be used (or according to the defaults), merged into an application as regular routes. To make sure they don't cause any confusion, they were merged at the very end. Oops. Strike one!

The following order was created:

Try to match a user route
Try to match a static file route (Handler::File)
Try to match a rendered template (Handler::AutoPage)
Give up and render a 404 page

While it seems reasonable for user-defined routes to trump static resources and templates, people were not expecting it to happen, which led to a few peculiar situations.

Problem 1: Greed is NOT good

In order for these routes to accept any input, they use a greedy megasplat route, which captures everything:

get '/**' => sub {
    # I SEE ALL ROUTES
    # (as long as they are a GET request)
};

(Users can define these routes as well, of course.)

The first manifestation of the reshuffling in order of execution occurred when users defined greedy megasplat routes which accidentally captured all static requests as well:

get '/**' => sub {
    # handle what I think is a user request
    # but can also be a static file request
};

Now users had to account for static files being asked of them. Ouch. This is actually the small problem.

Problem 2: A hook to kill

Remember hooks? Sure you do. Every route can have hooks. Amongst available hooks there are the before and after hooks which control the execution sequence of a matched route, allowing you to call code before and after it is executed, respectively.

How does that come into play? Since the File and AutoPage handlers create regular routes (just like a user might), the before and after hooks actually apply to them.

What?

Yes! This meant that, for example, your before route hooks were called for static files. If they contained a security check (for which many users use them), you might accidentally prevent a file from being rendered. Strike two!

"You can't see the lacking access picture because you lack access!"

Plack strikes back

Before we hit a strike three we decided to fix this. Since we're depending on Plack and the associated technology to help resolve so many issues, we looked there.

One wonderful part of Plack is the middlewares. One specific middleware that came to our rescue was Plack::Middleware::Static. It allow us to render static files from the public/ directory before it actually reaches our application and Dancer2 dispatching code at all. Wait, what?

The Plack middleware wraps the Dancer2 code and checks for static files on each request before it sends it off to Dancer2. If it can find a file that matches the request, it will serve it directly instead of passing it onwards.

This means we're back to a proper static file serving behavior. They are now served properly before the app and the hooks do not apply to them.

We still kept the File handler to provide send_file, but it is no longer in charge of automatically serving static files.

The AutoPage handler is kept as is because hooks make sense in rendering templates. You might want to use the before_template hook in order to provide default variables that will be available in every template rendering (including automatic ones served by AutoPage) - which is something you will not have with static content.

Conclusion

Wonderful work by Russell @veryrusty Jenkins solving this thorn in Dancer2 and providing consistent behavior that makes sense.

If you meet him, offer him a drink. He's Australian so he'll take it. :)

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Packing the Fat

Fri, 5 Dec 2014 00:00:00 -0000

Packing the Fat

After speaking of one way to handle dependencies, it's time to talk about another.

One promise we've made with Dancer2 is it will always be fatpackable, but only a few people actually know what this means and why it's important.

If you keep on reading, you're going to join that group of people.

FatPacker

One of the reasons people worry about dependencies is not being able to install them. Of course you could use local::lib or carton to install them locally, but another mechanism available is App::FatPacker.

App::FatPacker (using its command line interface, fatpack) packs dependencies into a single file (hence the name), allowing you to carry a single file instead of a directory tree.

If you're interested in understanding App::FatPacker, I have written an article describing it, how it works, and how to use it.

FatPacking Dancer2 applications

By saying Dancer2 is fatpackable, we mean that we make sure the Dancer2 can be fatpacked. We do this by requiring by default only pure-Perl modules. We allow you to optionally install XS modules to improve speed, but Dancer2 will work just fine without them.

This means that as long as your application is also pure-Perl, you could create a single file with your application and all of Dancer2 in it.

Let's take a look at an example.

Assuming we have an application in lib/MyApp.pm:

package MyApp;
use Dancer2;
get '/' => sub {'OK'};
1;

And we have a handler in bin/app.pl:

use strict;
use warnings;
use FindBin;
use lib "$FindBin::Bin/../lib";
use MyApp;

MyApp->to_app;

To fatpack it, we will begin by tracing the script:

$ fatpack trace bin/app.pl

This creates a fatpacker.trace file. From this we create the packlists:

$ fatpack packlists-for `cat fatpacker.trace` > packlists

The packlists are stored in a file called, surprisingly, packlists.

Now we create the tree using the following command:

$ fatpack tree `cat packlists`

The tree is created under the directory fatlib.

Now that we have the tree, we simply need to pack it all together. While this will create a file containing the dependency tree, we also want to add our script to it, so we'll do it all in one command:

$ (fatpack file; cat bin/app.pl) > myapp.pl

This creates a file called myapp.pl with everything in it. But before we can run it, we need to account for one small detail. Dancer2 is using MIME::Types which has a database of all MIME types and helps translate those. The small database file containing all of these types is a binary and therefore cannot be fatpacked. We need to copy it to the current directory so our script can find it.

$ cp fatlib/MIME/types.db .

Now we can use the file just like any other PSGI application file:

$ plackup myapp.pl

While we're using plackup here, it's just an example of a server that will read this. You would usually configure this in your web server as a PSGI application script.

Conclusion

Dancer2 aims to allow you to package your apps in any number of ways and App::FatPacker is a very useful packaging mechanism we aim to continually support.

You can now easily pack everyting and send your co-worker/colleague/enemy a single file that has everything in it, all dependencies bundled in.

Next time someone says "But I can't install anything", throw your packed Dancer web application at them!

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Your website in a carton box

Thu, 4 Dec 2014 00:00:00 -0000

Your website in a carton box

My new favorite way to package Dancer2 applications is Carton. You setup a single file, type a command or two, and you're done. Here's how it works and how you set it up yourself.

What Carton does?

Carton sets up a local copy of your project prerequisites. You only need to define them in a file and ask Carton to download all of them and set them up.

When you want to deploy your app, you just carry the git clone and ask Carton to set up the environment again and you will then be able to run it.

The benefit is multi-fold:

Local directory copy
By putting all the dependencies in a local directory, you can make sure they aren't updated by someone else by accident and their versions locked to the version you picked.
Sync versions
Deciding which versions of the dependent modules your project needs allows you to sync this with other developers as well. Now you're all using the same version and they don't change unless you want update the versions you want. When updated everyone again uses the same new version of everything.
Carry only the requirement, not bundled modules
Instead of bundling the modules, you only actually bundle the requirements. Carton builds them for you when you need it.

Setting it up

First thing we do is set up a new app:

$ dancer2 -a MyApp
...

Then we delete the files we don't care about:

$ rm -f Makefile.PL MANIFEST MANIFEST.SKIP

Now let's create a Git repo:

$ git init && git add . && git commit -m "initial commit"

Let's add a requirement using the cpanfile format:

$ cat > cpanfile
requires 'Dancer2' => 0.155000;
requires 'Template' => 0;
recommends 'URL::Encode::XS' => 0;
recommends 'CGI::Deurl::XS' => 0;
recommends 'HTTP::Parser::XS' => 0;

We can now ask Carton to set it up:

$ carton install
Installing modules using [...]
Successfully installed [...]
...
Complete! Modules were install into [...]/local

Now we have two files: cpanfile and cpanfile.snapshot. We add both of them to our Git repository and we make sure we don't accidentally add the local/ directory Carton created which holds the modules it installed:

$ echo local/ >> .gitignore
$ git add .gitignore cpanfile cpanfile.snapshot
$ git commit -m "Start using carton"

When we want to update the versions on the production machine, we simply call:

$ carton install --deployment

By using --deployment we make sure we only install the modules we have in our cpanfile.snapshot file and do not fallback to querying the CPAN.

Conclusion

Setting up Carton takes two minutes and allows us to provide specific requirements for our applications and a mechanism to install them on a production machine while ensuring it will create a local copy of everything.

We no longer care about the system module versions and we can maintain multiple versions of each module used in each separate application. If a certain application doesn't need to update a module, it just doesn't have to, and at the same time doesn't prevent another application from updating its own modules.

Try it out, you'll love it!

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Migrating to Dancer2

Wed, 3 Dec 2014 00:00:00 -0000

Migrating to Dancer2

A question we often get asked when we promote Dancer2 is "How do I migrate my code to Dancer2?"

There had been some changes between the versions - big enough to require an entirely new namespace. What were they? How do we migrate?

Changing the namespace

The reason we changed the namespace wasn't really due much to the change a user will necessarily see, but more due to the incredible difference in core architecture, and CPAN's inability to maintain two running versions in the same namespace, as mentioned in the previous article.

Dancer2 is actually a complete rewrite. It had small portions of the code moved and some of the documentation was copied over, but mainly it is completely restructured.

It no longer relies on reinventing the wheel as much. Users simply didn't care about installing another module or two or three or even more. At the same time, enough tools we written to still allow you to get away with not being able to install dependencies, so the entire problem became moot.

For example, Having local::lib allows you to install an entire module tree in a local user's directory (not requiring administrative privileges). Having fatpacker allows you to fatpack your entire Dancer2 application in a single file. Having Carton allows you to package it all nicely. Having Pinto allows you to set up an entire personal CPAN for your company. Things are much easier.

(Watch for additional articles that cover fatpacker and Carton in this advent calendar.)

Additionally, we wanted the ability to make core changes without breaking a lot of old applications. This doesn't mean a migration is difficult though. Having two different namespaces allows you to keep your current Dancer 1 applications running alongside your new Dancer2 applications and make it simple to migrate when you are ready.

Following are a few tips for migrating your applications to Dancer2, finishing with one great resource for it at the end.

It's all about the Plack

Dancer originally wasn't built for Plack. When Plack came out, Dancer has made the necessary changes to accommodate it. However, it wasn't the leading spec for Dancer, as it was simply a port of another framework from Ruby.

Dancer2 took a different approach: Plack leads the way. This led to quite a few changes in removing unnecessary code, producing much more correct behavior, and providing more deterministic results.

A few examples:

Each class is an entire app
A Dancer App is now every class which consumes Dancer2. This means they can each have their own configuration, allowing different engines:
```
package Admin {
    use Dancer2;
    ...
}

package Public {
    use Dancer2;
    ...
}
```
Now Admin and Public are fully independent applications. They will not share any resources and will be able to have different engines (template, session, serializer, logger) which will not be shared. No more worrying about globals.
Removing ':script' import key for plackup
Dancer 1 had the :script import key which would read your command line arguments to change how the Dancer server would run.

Since Plack has the plackup command line tool, it already reads your command line arguments. Why make a duplicate effort?
Fully object-oriented
Dancer2 has changed its core substantially to be much more object oriented. Now the core is implemented in objects. The DSL layer simply provides an interface to those objects.

This makes everything much easier to compose and test.
HTTP::Server::Simple replaced with HTTP::Server::PSGI
Originally Dancer would use HTTP::Server::Simple which supported PSGI, but it was a pain to use, since it had issues we needed to patch and provided a dependency we weren't happy about.

As it happens, Plack comes with a great development server called HTTP::Server::PSGI. Since we're building on Plack, we might as well make use of it. This is why Dancer2 uses it instead.

Migration in a nutshell

Okay, but putting all of that aside, how do I migrate?

Well, it really depends on what you do with your application, but here are our main migration tips:

Clean importing
All of the importing keywords are unnecessary now, except for appname, explained below.
Separate serialized routes
Routes that require serializers should be separated to their own Dancer2 app, since serializers will try to deserialize all input and serialize all output in every route in their app scope.
DSL keyword changes
We tried to change very little of the DSL. load and param_array no longer exist, and session can also provide the session object, not just retrieve and set values for keys.
Applications are self-dispatching units
As explained above, any module that uses Dancer2 becomes a Dancer Application, which creates a separated dispatcher only for the routes created under that module namespace.

The application's engines will not be shared with others - an issue that pained experienced developers of Dancer 1.

If you want to compose multiple Dancer Applications to create a single web application, you can use Plack::Builder and to_app. Stay tuned for an article explaining this exact situation.
Multiple packages for a single app
Allowing multiple packages to create a single Dancer Application is also possible using the appname feature. This will also appear in an upcoming article.

You said something about a great resource?

Oh yes!

Writing down all the migration changes between Dancer 1 and Dancer2 would be too much. Plus, who would check this article after a while again to see if it was updated? No one, that's who!

Instead, we've compiled a list (with the kind help of Snigdha Dagar) of all of these changes under Dancer2::Manual::Migration.

If you're interested in migrating your applications to Dancer2, or would like to write a new application and care to know what the differences are, you should consult it.

Of course we welcome any changes and improvements to this document as it is a living breathing document (which we hope will not attack us during Halloween), so feel free to submit pull requests, send comments on the mailing list, or stop by the IRC channel.

Conclusion

Dancer2 has a few changes that promote much more consistent and deterministic web application development. It reduces the considerations we have to take into account while providing much more correct implementations for various pieces.

Our new Migration Document helps new-comers and seasoned developers keep up-to-date with the changes we've done and how it might affect their future code.

Author

This article has been written by Sawyer X and Mickey Nasriachi for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Dancer2 - starting anew

Tue, 2 Dec 2014 00:00:00 -0000

Dancer2 - starting anew

It has a different namespace, a different release manager, and a different repo - Dancer2 sounds like a completely new thing, which it is. But it also isn't.

The portsmith's history

Dancer started as a Perl port of Sinatra - a successful light web framework in Ruby. The author of Dancer, Alexis Sukrieh (AKA, sukria), who is also a Ruby programmer, had enjoyed it so much, he wanted something similar in Perl. He then began the task of porting Sinatra to Perl under the name Dancer.

Dancer had become very successful. In fact it had been so successful a wonderful warm community grew around it. Users and developers alike sharing their use cases, giving talks, trainings, writing articles, and even making it part of their job.

When Dancer became a production framework in various companies, a major flaw in the design had creeped into sight: the global state. (Chime in with horror music.)

Many components in Dancer were globals and this shared state made it impossible to have multiple applications without side-effects. Sukria had gone underground and returned with Dancer 2. A completely rewritten framework, based on the same principle as Dancer, sharing the same DSL and overall design, but lacking the global state which had crippled Dancer so.

This version promised to be the noble successor.

The version that never was

Unfortunately the code was still incompatible in some ways. Plugins, mainly, did not work, since the architecture was vastly different. We wanted to release both a 2.0 branch and a 1.0 branch, but CPAN does not allow that. We then decided to split the namespace to allow both versions to exist at the same time while we iron out the missing bits. Considering Mongrel2 and even Perl's Amon2, why not Dancer2?

After a few releases, Sukria had to attend to more pressing issues, and Dancer2 received little attention. Sawyer, who had become the project leader for production version of Dancer, took Dancer2, and Yanick Champoux, the amazing French Canadian, had taken the production version of Dancer under his wing - one of many wings, in fact.

The prodigal daughter

Following multiple big releases, Dancer2 had become ready for mass usage. It has a few rough edges, but in most ways, it's in fact much more reliable.

Dancer2 has the following advantages over Dancer 1, amongst others:

Composable Dancer Applications
Dancer2 Apps are composable units that take care of their own dispatching. While we have the original dance behavior to dispatch over all registered applications, we now have much better mechanisms to handle separate applications without forcing the user to have all or nothing.

This Advent Calendar will feature an article showcasing this functionality with plenty of explanations on the matter.
Decoupled DSL implementation
The DSL implementation in Dancer2 is decoupled in a way that allows us to improve it without changing it for the users. In fact, Dancer2 shares the majority of the DSL with Dancer 1 and we can keep that working as expected.
Successful object system
Instead of rolling our own objects, Dancer2 uses Moo, which provides the same behavior as Moose for all the objects we have. It allows us to have much smarter objects than we did before with lazy attributes, attribute builders, role composition, and more.
Better development server
Dancer2 uses the Plack core development server, HTTP::Server::PSGI, which provides a more reliable experience. In the past, for Dancer 1, we had to work around a lot of problems with the development server.
Less specialized code
Dancer 1 contained a lot of unnecessary code which was removed in favor of a proper implementation around Plack: this includes the command line parsing when importing the DSL, static file serving, testing code, and more.
Robust CLI capabilities
The command-line interface in Dancer2 is written in App::Cmd, which allows us to extend it fully, providing interesting command-line features.

We haven't yet gone nuts with it, but now we'll be able to do so, and we have a few interesting ideas.
Devoted set of developers
While Dancer version 1 has been placed into freeze mode, Dancer2 has a group of developers devoted to it. More and more developers have joined the core team as well. This includes the aforementioned Yanick, Russell Jenkins, Mickey Nasriachi, Stefan Hornburg (Racke), Steven Humphrey, Alberto Simões, and David Precious. Somewhere in there Sawyer X is probably there, but we can't attest to that - at least not legally.

At the end of the day, though, there is one major consideration we've yet to mention.

The version that is

While many users are still on Dancer 1, Dancer2 is ready and waiting, and we're pooling all of our efforts to improving it. Dancer 1 is on freeze mode, meaning no new features.

Dancer2 is NOT a new framework, it is simply the next version. When asked in the past, we explained that this isn't Perl 5 and Perl 6.

Dancer2 might have a few rough edges, but it's much more reliable, correct, featureful, and a much safer bet than Dancer 1.

Conclusion

Please consider writing your new code in Dancer2 and also porting any major applications you want to continue to develop. If you find difficulties in the process, please let us know, so we could fix them.

Be careful though, that's the first step to becoming a contributor! :)

Author

This article has been written by Sawyer X for the Perl Dancer Advent Calendar 2014.

Copyright

No copyright retained. Enjoy.

2014 // Sawyer X <xsawyerx@cpan.org>

Another year of Dancing...

Mon, 1 Dec 2014 00:00:00 -0000

Another year of Dancing...

Welcome to the Dancer advent calendar!

We skipped last year's advent calendar, but we're hoping the articles we gathered this year will help you learn more about Dancer and specifically Dancer2, which is the target version of all new applications.

We will show changes Dancer2 has made, features it recently added, a few tricks, and even survey the code - as well as mention some of our numerous community contributors, and take a look at the project as a whole.

More and more dancers...

Usage of Dancer continues to grow, with more new users added to the dancefloor.

If you're using Dancer in your company and would like to appear on the dancefloor page, please contact us.

Dancer has been spotted as a required/desired skill in several job adverts, showing the increased uptake of Dancer in corporate web apps. Try searching for Dancer on jobs.perl.org.

Dancer2 - a reality

Far more progress has been made with Dancer2. It is now a fully usable, robust replacement for Dancer 1. That said, Dancer 1 will continue to be supported for some time yet and continues to receive bugfixes, support, and minor improvements.

This article will focus mainly on Dancer2.

Dancer at LPW2013

Andrew Solomon ran a free two-hour Dancer 2 training course at last year's London Perl Workshop. Andrew's course was a hands-on training session to develop a website with dynamic content using Dancer, during which the attendees would:

Learn about Dancer2 as a framework and as a language
Implement route handlers using sessions and hooks
Implement views, templates, and layouts using Template Toolkit and Bootstrap
Understand the concept of Model-View-Controller
Experience structuring code for maintainability
Experience using object oriented Perl modules
Understand all the crazy jargon above!

See also Andrew's Perl Web Development course on geekuni.com.

DancerConf

This year in Hancock, New York, the first Dancer conference was held. It had trainings on Dancer (and Dancer2), DBIx::Class, and other related technologies.

We plan on having another conference in 2015 as well.

Dancer IRC channel community grows

The community of helpful users on the Dancer IRC channel continues to grow! You can find us on irc.perl.org in #dancer or irc://irc.perl.org/dancer if your system is configured to support irc:// links.

If you do not regularly use IRC but wish to join us, there is also a web IRC client available at http://dancer.pm/irc for ease of use.

AUTHORS

David Precious (BIGPRESH)

Sawyer X (XSAWYERX)