by ·0 Comments

Watching the cricket today, I was wondering what a graph of the different teams’ runs per over would be, over the past decade or so. The English team seems to bat at close to 4 an over lately, while the South African team seems happy at a more ‘traditional’ rate of around 3 per over. Maybe because the English have played the Australians in the Ashes so much, they have started to emulate their run scoring pace – that was my hypothesis. It was the Aussies, after all, that really ramped things up 10-15 years ago, when they dominated test cricket for a decade.

I wanted to check my theories against the data, so I downloaded the test data from Cricinfo StatsGuru. Checking the team scoring rates (RPO) year-by-year had too much variance, so I adjusted the graph to use the run rate from the 5 previous years. This is what you get:

It’s quite impressive to see how the Australians broke the mold. South Africa is pretty consistent, but England aren’t that much faster than the Proteas as I thought they’d be, at least not over the last 5 years. New Zealand’s black caps seem to be cranking it up, though. The Zimbabweans haven’t yet reached the levels most teams were at in 1999 (their hiatus shows in the 5-year average as a gap in 2010).

Using the run rate from the previous 10 years, the graph smooths out a bit:

You can just see the passing of the baton from the West Indies team to the Aussies – it would be interesting to run this even further back than 1990, which was as far back as I went. One day, when I’m big, I’ll have to redo it with all test data, and make it an interactive webpage. For now, the above are images from an Excel spreadsheet…

But it made me wonder: RPO shows how fast the team bats, but what would the batting average (per wicket) look like?

In the 10 years up to 2009, an Australian wicket scored an average of roughly 10 runs more than a wicket from all the other teams! Nice to see the South Africans getting to the top, although as the 5-year average shows (below), it looks like we’re on a bit of a decline.

Nice to see how Bangladesh is now competing with the older test playing nations. Are Aus really still top over the last 5 years? Maybe their bowling lets them down, as a team. I suppose the same exercise needs to be done for the bowling team… not tonight, though.

by ·0 Comments

Lately I’ve been building websites with AngularJS, TypeScript, WebApi and Entity Framework 6 (and Bootstrap, obvs!). A lot of the work is repetitive grunt-work, generating the model, then the DTO, then the WebApi controller, etc. For a little while now I’ve been using my own “code generator” tool to scaffold out these “templates”, which saves me a lot of work. Recently I open-sourced the project on GitHub, and now I’ve got a chance to blog about it.

I’ve also set up a hosted version, which you can play with now, to see how it works. Simply head over to http://codegenerator.sitedemo.co.za/ and log in with the following credentials:

  • Username: demo@capesean.co.za
  • Password: L3tM3!n

Here follows an explanation of how to use it, and what it does.

Projects

When you first log in, you will see a list of projects. Click the Demo Project for now. You will see a page like the following:

You can see there are 4 entities: Product, Customer, Order and Line Item (we all know where this is going, right?)

You’ll see 13 columns in the table: Model, Enums, DTO, SettingsDTO, etc. These are the outputs that will be generated by the tool. More on these later.

Entities

Clicking on the Customer entity link, takes us to the following webpage:

Some entity-level fields are hidden under that More > button, but we’ll ignore those fields for now. More importantly you see the 3 fields on the entity: CustomerId, Name, and Telephone. You can add more fields, rearrange the order, or click on a field row to edit that field.

Beneath the fields are the relationships with other entities: those relationships where the Customerentity is the parent, and then where it is the Child. Customer is a parent of Orders, so you can see the relationship to the Orders entity listed.

Beneath that you’ll find Code Replacements. Code Replacements allow you to customize the generated outputs. More on that in another post, though.

Fields

For now, let’s click the Name field. This brings up the following page:

Here you can see some standard things you’d need for a field: a field name, a label, the data type (e.g. nVarchar), the Length, whether it’s a key field, if it’s unique, if it’s nullable, whether it’s a search field, whether it should be shown on the search results page, a sort order, etc. So enough to get you started.

The Code!

Back on the entity page, if you click the blue Code </> button, you’ll get to this page:

Here you’ll find 13 checkboxes and 13 tabs: one for each of the output files that the Code Generator tool produces. (The checkboxes are for deploying the outputs directly to your local folder, if/when the tool is installed on your development machine – again, more on that in a later post).

The Model Code

For now, you’ll see that the tool is generating a Model file in the screenshot above. It’s outputting the key field, CustomerId, which is a Guid. The Name field is a 250-length string, with an index for uniqueness. There’s also a navigation property to the Orders collection, which is produced because of the relationship defined from Customers to Orders.

The WebApi Controller Code

Let’s look at the WebApi controller code next:

So the API is protected with Authorize(Roles = “Administrator”), and the route prefix is api/customers.

There is a Search end-point which takes an optional paging object, for paging through results, and then a string search parameter q. If this is supplied, the controller will search in the Customer.Namefield for any matches, because the Name field was defined as a text-search field. The name field was also designated as a sorting field, so the results are sorted by the Name. The controller then gets a paginated response object, and converts the model to the DTO using the ModelFactory.Createmethod.

Further down you’ll see a Get method, for returning a single item. Then an Insert and Update method, which both use a private Save method, and lastly a Delete method.

The AngularJS (TypeScript) Code

Ok, let’s look at what it does from the AngularJS / TypeScript side. Here is the output for the AngularJS controller, for the Edit page:

Ok, so this is a standard AngularJS controller, with several items injected. Let’s look at what it does.

There’s an initPage function which runs when the controller loads. It determines via the $stateParamsif the entity is being added (new) or loaded (existing). If it’s being loaded, it uses the customerResource (ngResource) to .get the appropriate record. That’s really it, in a nutshell.

Then there’s a save function, which saves changes up to the API.

Then there’s a delete function, which will delete the entity.

And lastly there’s a loadOrders function, which will load the customer’s orders, using the pagination parameters that were mentioned briefly in the Controller section above, so it will display 10 orders at a time with a pager to move through them.

The Html Code

Lastly, let’s look at an Html page that gets output.

Here’s the Html that works with the AngularJS controller. I’m not going to go into detail, but you’ll see it uses Bootstrap 4 (although there’s a setting for 3 on the Project), it does a bit of validation and uses ng-messages, has the Save and Delete buttons, and then displays the customer’s orders in a list at the bottom (if the customer record is not new).

Hopefully that explains it enough to show you what it can do. Obviously you’ll need to have a project with the appropriate supporting files (e.g. the BaseApiController.cs, the WebApiConfig.cs, the ApplicationDBContext.cs, etc). However, the CodeGenerator project on GitHub has all these files in it! So you can simply strip out the files related to the CodeGenerator, and paste in the files from your project, and you should be good to go. (If you’re struggling with getting that set up, I can/will provide an ’empty’ project that you can start with, if it would help.)

Let me know if you find it useful, and if you hit any issues. Hopefully this will help someone’s productivity as much as it’s helped mine!

by ·3 Comments

Recently I’ve started using Azure for my applications and their database. One of the first things I encountered was how slow Azure SQL DBs seemed to be, compared with performance on my local machine and on other hosting services I’ve used. A query I was running on a dev laptop, which isn’t a beast of a machine, would regularly take under a second to complete. The primary table it was querying has about 200,000 records, and there were about 5 or 6 joins.

When the same database was up on Azure, my website kept timing out when it would hit that query. I ran the query through Management Studio and reliably got 42 second response times. This was running on an Azure S0 database instance.

Searching the web for “slow azure db”, I came across this result:
https://feedback.azure.com/forums/217321-sql-database/suggestions/6848339-please-reconsider-the-new-db-pricing-tiers

Seems like I’m not the only one, and that with the change in pricing last year (which predated my Azure experience), the performance of relatively small databases has declined significantly. Most people who posted on the above page were saying that they had to upgrade their instances to get (not significantly) better performance. This obviously comes at a cost, but I thought I would try it out.

I upgrade to an S1 instance: the query time was reliably 20 seconds.

I upgraded again to an S2 instance: the query time was reliably 12 seconds.

Clearly the performance was improving, but the cost was too. I couldn’t justify the cost of an S2 database. It’s not exhorbitant, but it would have been an unpalatable cost to my customers to have to charge them that much, when other hosting options are so much cheaper. And 12 seconds is still nowhere near good enough – it needed to be under a second, otherwise my website just wouldn’t be fast enough.

As a last resort I tried adding indices for the query. I hadn’t thought of doing this because the query ran so fast on my local machine, and because all the joins were on the foreign tables’ primary keys, I thought the query was “good enough” and that the difference in performance between 0 seconds and 42 seconds couldn’t possibly be an index issue – surely Azure should give comparable performance to a mid-range laptop or another hosting option?

I was still on the S2 instance when I tested it with the indices in place. Query time was 0 seconds!

I decided to downgrade back to the S0 instance. Query time remained 0 seconds!

Fantastic! If Azure SQL needs the appropriate indices to get it to perform well on the lower instances, I can handle that.

So here’s my thinking of why it works like that, although I’m no expert: On my laptop, and on the other hosting provider, if a “big” query comes along, all the computer’s/server’s resources (RAM/CPU) are used by that process and the query is resolved quickly. That’s fantastic unless your site is sitting on a shared server where other websites are generating heavy workloads, and your simple queries get queued until the resources are available again. I read that the changes in pricing model on Azure were partly precipitated by complaints about the predictability of Azure DB performance.

My guess is that this is what was happening on Azure previously. The change meant that you are now (with the eDTU pricing model) practically guaranteed a level of performance. That’s an up-side and a down-side: your database is not going to be able to consume huge amounts of resources to process (relatively) expensive queries. The solution is to make sure your queries are tuned as much as possible; if you have the appropriate indices in place (which you wouldn’t notice the lack of during development), you should still be able to get decent performance from your Azure SQL database, at an acceptable price point too.

 

by ·69 Comments

In this post, I’ll show how I set up ASP.NET5 (vNext / ASP.NET Core / MVC6 / ASP.NET MVC Core) to use JWT tokens.

29 Jan 2016 UPDATE: The Github repo has been updated to include an AngularJS login.

First up, let me say I’m not expert on any of this, but if you see any room for improvements / errors, please let me know.

TLDR: If you don’t want to read the whole thing, you can download the finished code from: https://github.com/capesean/openiddict-test

The website I will be using this on is an AngularJS front end (although that’s not really relevant), which I hope to one day build mobile apps for, so JWT tokens seems like the logical choice. My understanding is that I’d be using the Resource Owner Flow of OAuth2; and that my authorization server (the thing that creates the token) and my resource server (the thing that validates the token and returns the data – i.e. the API) would be in the same application. Also, users wouldn’t (initially) be authorizing other apps/websites/etc. (i.e. “clients”) to use their data (resources) – so the typical authorization aspect that you see in many OAuth2 examples (“Application X wants to access your data – Accept / Deny?”) didn’t apply. Basically, I just wanted a typical login (i.e. cookies) system for round trips to the API, but without the cookie – i.e. with JWTs!

Note that I use Entity Framework and SQL Server for data access/storage, and I’ll be using ASP.NET Identity v3.

First I needed to choose which existing software to use to generate the JWT tokens. Previously, in ASP.NET4, I had used an OWIN implementation that I based on the fantastic blog of Taiseer Joudeh – at www.bitoftech.net. It involved quite a lot of coding so I was hoping for something in v5 that had less heavy lifting.

I looked at Identity Server v3 first, and got reasonably far, but it also got quite tricky to understand, as you have to look at an example for using Identity, and then another example for Entity Framework, and then another example for ASP.NET5, and then another example for JWT tokens, etc – and then try pull them all together. I gave up when I read something about Entity Framework 7 not yet being fully (natively?) supported.

The other option I was reading about was AspNet.Security.OpenIdConnect.Server (ASOS). But again, this looked quite tricky to pull together. then I stumbled onto the lesser known OpenIddict software, which is based on ASOS, but does most of the heavy lifting for you. This is what I ended up choosing.

There are installation instructions on the OpenIddict home page. Note that at the time of writing, you have to use RC2 builds, so you’ll need to run:

You also have to create a Nuget.Config file in the root of your application, with the following contents:

At the time of writing, these installation instructions mirror what’s on the OpenIddict home page. From here on we’ll start to deviate, though, as we don’t want the default configuration, which seems to be more about authorizing other applications/clients.

First up, we add hosting.json to the project with the following contents:

My project name is openiddict-test, so in project.json we set the following commands:

Note that the web command is the project name, not the namespace name (I originally had it as openiddicttest and the damned thing wouldn’t start).

The new static void main syntax in Startup.cs is now as follows:

In the config file (in my case, in appsettings.json), we have the database connection string and some logging settings:

Lastly (before we get into the real code), in project.json, make sure you have the following dependencies:

We don’t actually need the whole of OpenIddict, we just need the core for generating the tokens, and the EF for the entity Entity Framework aspects, so I’ve just included these two, although you could include the main package if you like. We’re also going to be using the Microsoft JwtBearer package for validating the tokens, so include that now too. Notice that all dependencies are using the “-*” versioning option.

The Real Code

This is where it starts to get interesting.

First we need the User model, so this is the ApplicationUser.cs file:

Note that we’re deriving from IdentityUser as you normally would if using ASP.NET Identity. We’re also adding a custom field called GivenName, which we will include in the JWT token.

Similarly, here is the very simple ApplicationRole.cs file:

It simply derives from IdentityRole.

Then we need a DbContext, so the below is from a file ApplicationDbContext.cs:

Note that we’re deriving from OpenIddictContext and we’re using an override with 4 parameters: the ApplicationUser, the Application class, the ApplicationRole, and string. Although we’re not going to be using the Application class, we still need to use this signature otherwise the roles won’t get populated in the JWT token.

Now, here is the ConfigureServices method from the Startup.cs:

  • Lines 4-7 are standard for building the configuration options.
  • Lines 10-13 adds Entity Framework and configures the connection string from the configuration file.
  • Lines 16-19 adds Identity. Notice Line 19 being the first OpenIddict line of code, which also tells OpenIddict to use Entity Framework.
  • Line 22 adds MVC, for the API.
  • Line 25 adds the Database Initializer, which is simply to seed the database with some data.
  • Line 26 adds a Custom OpenIddict Manager. The OpenIddict Manager will override the token generation method so we can add the custom field (GivenName) to the JWT token.

Let’s look at the CustomOpenIddictManager.cs:

This class simply inherits from OpenIddict.OpenIddictManager<ApplicationUser, Application> and then overrides CreateIdentityAsyncCreateIdentityAsync. It creates the claimsIdentity from the base method, and then adds the Given Name claim to the claimsIdentity, and sets the destination property of the claim to both id_token and token. This simply means that when the token is requested, the Given Name claim should be returned in the token.

Back in the Startup.cs file, we now look at the Configure method:

  • Lines 3-5 set up logging.
  • Line 7 has something to do with IIS, I won’t pretend to know what.
  • Line 11 & 10 allow static files to be served (index.html) and lets the index.html page be served as the default page (index.html will be our test javascript page).
  • Note we don’t use the identity line on line 14 as this is just a wrapper for cookies.
  • Lines 16-23 configures OpenIddictCore – telling it to use JWT tokens, to allow non-HTTPS requests, and to display errors (these last two are obviously development settings).
  • Lines 26-33 configures the JWT middleware that will validate the token and thus ensure that the Authorize attribute on our WebApi methods is honoured. Note that the audience and authority are the same, but more importantly the audience must match the resource value sent in the token request payload (on the index.html page, which we will see later). Also note that you will have to put in the correct values for your environment here and on index.html, and that this should ideally be stored in the config file.
  • Lines 36-41 configures MVC.
  • Line 44 seeds the database.

I’m not going to cover the Database Initialiser as that just seeds the database with a user and a role. You can view the source on Github if you’re interested.

The API endpoint we will be calling to test the token is in TestController.cs:

This is a pretty standard controller, with these notes:

  • On line 8 we are using the Authorize attribute to ensure that only authorized (logged in) users can access this data.
  • On line 9 we’re inherting from the MVC Controller class.
  • On lines 11-18 we’re instantiating the controller object, using Dependency Injection to get the ApplicationDbContext and the UserManager, which we store in local properties.
  • Line 20 specifies the Route (api/test) and that it’s a GET request.
  • Line 23-26 gets the user that is logged in. If there’s no user  it returns a simple text message (there always will be a user if the Authorize attribute is in place on line 7, but you can remove the attribute and call the url to test it’s working). If there is a user, the user object gets returned.

Basically, this API url (http://localhost:58292/api/test) will return the user if logged in, or it will return 401 (Unauthorized).

That is really all there is to it, on the server side.

Client Side Testing Code

This post is not meant to cover what happens on the client side – it could be an AngularJS website, or a mobile app, or whatever. But to test that the server is working, below is the index.html test page:

  •  Lines 9-21 are the form fields for logging in, and the buttons for logging in (getting the token) and getting the api data (using the token).
  • Lines 22-27 are the outputs of the results of the two button clicks.
  • Lines 29-32 sets up the base url and the button click event handlers.
  • Lines 58-104 gets the token (logs the user in). It basically creates an XMLHttpRequest that when loaded, will output the token to the controls in lines 22-27. You can ignore the client_id and client_secret lines. Note that the url being called is the base url plus connect/token. Note the grant_type, the resource (which, as mentioned ealier, must match the settings in Startup.cs), and the scope (offline_access is required to get the refresh token).
  • Lines 34-56 access the API url. It creates an XMLHttpRequest that will log the results to the controls in lines 22-27. The API url is simply the base url plus api/test. Note that the access token is sent as a Bearer token (“Bearer ” + access_token)  in the Authorization header.
  • Lines 106-132 simply decode the token so the contents can be shown on the page.

If all goes well, you should see the result of the token request (login) and the data request (api call) as shown in the screenshot below.

The code is at Github: https://github.com/capesean/openiddict-test

Good luck!

jwttoken

 

I must thank Kevin Chalet, aka Pinpoint / PinPointTownes for all his assistance. Kevin is the author of OpenIddict and had endless patience with me while I was fumbling my way around.

by ·0 Comments

I’ve started working with AngularJS for any new websites, and one of the first things I needed to revisit was the authentication strategy. Pretty much every website I build requires the user to be logged in to access (at least part of) the website. I wanted to put together a ‘template’ that I could reuse each time I start a new project. TLDR: the end result is up on github at: https://github.com/capesean/JWTKickStart After doing some research, I really like OAuth2 and JSON Web Tokens (JWT). I use .NET on the server, but one of the appeals of JWT and Angular was a complete decoupling of the browser app from the server’s API. As you’ll see from the Github repo, there are two Visual Studio projects:

  • APP: this is the AngularJS application. Although it’s in a Visual Studio project ‘container’, there is no dependency on Visual Studio at all.
  • API: this is the server back-end, which has been built using .NET’s WebAPI 2 with OWIN & C#. Although, as mentioned, this could be any API that issues JWT tokens & refresh tokens.

Before I start, my inspiration came from the following sources:

  • The BitOfTech blog by Taiseer Joudeh is an amazing resource for Angular/JWT/.NET/OWIN articles. Just amazing. Thank you, Taiseer! However, the main series of articles I drew from used OAuth bearer tokens, not JWT tokens. There are other articles that cover JWT tokens. I rewrote some aspects (e.g. the Refresh Token side) for JWTs.
  • I also really liked this article on the Brewhouse.io blog. It shows how to build a modal login that pops up if the user makes an API call which is not authenticated, giving the user a change to re-enter his/her credentials before re-attempting the API call, which is very slick. I extended this to use the Refresh Token before requiring the modal login.
  • In the comments on the Brewhouse.io article is a video from a talk by Martin Gontovnikas, from Auth0. He suggests a very similar strategy. Some new nuggets in there too, though.

Some of my requirements:

  • Typically my requirements won’t include multiple clients (website, mobile app, etc.), so I removed the multi-client aspect (where client data gets stored in the database) to keep the starting point simple. It should be relatively easy to add back: check out the BitOfTech articles if that’s a requirement.
  • Typically I wouldn’t need to cater for CORS either, so I initially left that part out. However, when I separated the system into the two projects (APP & API), that run on two localhost ports, then I needed to add it back. So it’s a pretty simple CORS * at the moment, but for production it’s easy enough to remove the CORS AppSetting so it falls back to a same-origin policy.
  • Typically I would have the Resource Server, the Authorization Server, and the Authentication Server as one & the same. I don’t need Facebook etc. logins (I only require username & password access), and my API backend handles everything: authentication, authorization, and the actual API itself. So I merged these all into one.

Some notes for setting it up to run:

  • A SQL Server .bak file is included in the API projects App_Data folder.
  • When it’s run, it will create a user with username ‘test@test.com’ and password ‘password’ in the database.
  • The token & refresh token Url is at /token
  • An authorisation protected resource is at Url /api/protected
  • Login details are prefilled in the login forms
  • Settings for the API project (e.g. Base64 encoded secret, connection string, Client Id, token expirations, etc.) are stored in web.config.
  • When logged in, you should see your jwt & refresh tokens outputted as JSON, and the response from the protected API call, which outputs the claims for the logged in user, as shown below: JWTKickStart
  • You can set the auth token to expire in a minute (1), and then log in: after a minute, the system will use the refresh token to keep you logged in. The neat thing here is, if your claims have changed, I’ve built it to update the ticket, so for example if your permissions change, the app will ‘silently’ get updated with your new status.

I’m going to (hopefully) maintain this repo, so it’s always a good boilerplate project to get started with. In the meantime, here’s the Url again: https://github.com/capesean/JWTKickStart I hope it helps someone!