I’ve been trying to figure out the cause of a SQL deadlock problem, and it dawned on me that preventing deadlocks shouldn’t be that hard. You just have to save data in the same order every time, and the easiest way to do that is to have a repository with one save method. I’ll show you what a deadlock is, and how to prevent them.
Category: C#
What’s the difference between Span of T and Memory of T?
What’s the difference between Span of T and Memory of T? My goal is not to tell you how awesome they are, but to get you over the hump of understanding how they work. I cover what they do and why, along with some description of their internals. Finally, I discuss what the owner consumer model is, and why it’s relevant to Memory of T.
Loading Millions Of Rows Of Test Data In Seconds
Most tools that generate test data do so iteratively. But SQL loves set based operations. With a little T-SQL know how, you can create millions of rows of test data in seconds. You can even leverage public data sources to create more realistic data that conforms to your applications business rules. No third party tools, just a handful of queries. I’m keeping it simple.
Accelerate By Sorting
A fundamental axiom of computer science is that you cannot make a computer go faster, you can only reduce the work it has to do. One of the most common ways to do less work is to sort the things your code is processing. While experimenting with different ways to populate a set of aggregate roots, I found that the simple act of sorting the results from the database could make a significant difference in performance.
Normally when I have a set of parent objects, each having a collection of child objects, I’ll use a foreach loop to iterate over the parent collection. A LINQ where predicate provides an easy way to filter which children belong to with their parents. The problem with this is that using the LINQ where predicate creates more allocations than are necessary, and it loops over all the child objects once for each parent object.
foreach (var customer in customers) { _setOrders.SetValue(customer, orders.Where(_ => _.CustomerId == customer.CustomerId.Value).ToList()); }
If my child collection was sorted by the parent identifier, I would only have to loop over the child collection once. Let’s look at what that looks like.
position = 0; for (var i = 0; i < customers.Length; i++) { while (position < orders.Length && orders[position].CustomerId == customers[i].CustomerId) { customers[i].Orders.Add(orders[position]); position++; } }
The counter is defined outside the for each loop, and does not get reset to the beginning of the child collection with each iteration of the parent collection. Doing this eliminates having to iterate over the part of the child collection that has already been covered. Once the element of the child collection doesn’t match with the current parent element, we can advance to the next parent element.
Using the nested while statement with a counter may reduce the readability of the code, but it’s not overly complex, and the performance benefit speaks for itself.
So how much of a performance boost do you get from using this technique? Obviously it depends on how many objects are in the collections. If there is only one parent and one child, it will be slower, but not by much. It doesn’t take long to sort a collection of one.
Using a collection of 100 parents, 1000 children, and 10,000 grandchildren. There was a 50X improvement. Iterating over 10,000 grandchildren 1,000 times makes a big difference.
The next question that should spring to mind is how relevant are the time savings when making out of process calls to a database. Again it varies, but with sample data, I’m seeing a 3X improvement.
My suggestion is if you identify a situation where you need a performance improvement that involves iterating over a collection to find set of items….sort it. The impact may surprise you.
Supercharged Dapper Repositories
Part #1: Separate Filters From Queries
Dapper has been a really popular data access library for over a decade, but when it comes to use it with domain entities, there are a few things that can create friction. In this series, I want to show you how to supercharge your Dapper repositories to be faster and more maintainable.
Sample Code: https://github.com/JZuerlein/DapperPopulateAggregate
Paging Data With SQL Server and C#
Learn about several strategies to improve the performance of paging data with .Net and SQL Server, as well as different ways to measure the impact of your software.
NPI data is available at CMS.gov https://download.cms.gov/nppes/NPI_Files.html
SqlConnection Statistics For SQL Server https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/provider-statistics-for-sql-server
Jeff Zuerlein https://www.linkedin.com/in/jeff-zuerlein-2aa67b7/
Traffic Report: Avoid the 403, Make Authorization Checks In Your UI To Avoid Confusion
Nothing disturbs the flow of automobile traffic like a sink hole big enough to swallow your car. Drivers need to navigate around it, and they become scared to death that another one will appear. HTTP 403 Forbidden error messages have a similar effect. When a user is happily clicking through your web page and suddenly get an error message, because they don’t have authorization to perform an operation, it has a really negative effect. They don’t have an understanding of why it occurred, so it’s up to you to prevent these type of problems.To provide a good user experience, UI elements that lead to an action the user can not perform should be hidden, modified, or never included in the page at all. The trick to making this easier, is to leverage the tools and code you have already put in place to protect Controllers and Actions in your ASP.NET Core application. Let’s take a look at using existing Authorization Policies and Tag Helpers when constructing Views.
IAuthorizationService
Authorization checks in ASP.NET Core can be made imperatively using an instance of the IAuthorizationService. It has a method called AuthorizeAsync that can take in a user, a policy name, and on optional resource to determine if the user has can perform an operation. This means that we can use the same policies that were defined in our StartUp.cs file to find out if a user will be able to execute an action in a controller. To make that happen, we’re going to need the same instance of IAuthorization that was configured in the Startup.cs file. The dependency injection system that comes with ASP.NET Core is also available in Razor views, so we can use it to get a copy of the authorization service. The @inject directive can be added to the top of a Razor page along with a variable declaration to get a reference to the object in the dependency injection system. If you need to include authorization checks on all of your pages, it may make sense to include the authorization service in the _ViewImports.cshtml file. Any using or injection directives you add to the _ViewImports.cshtml file will become part of any view that get rendered.
@using Microsoft.AspNetCore.Authorization @inject IAuthorizationService AuthorizationService
Once we have used DI to get an instance of IAuthorizationService, we can use it to perform authorization evaluations. Suppose we have a list of resources, and we want to display a link to each resource that the user can view. We could simply loop through the collection of resources and perform an authorization check passing in the resource. The AuthorizeAsync function found in the authorization service will perform the policy evaluation and return the result. We can use that result to decide if the link should be included in the page. For the example below, I’m using a resource that represents a server room, and the user must meet certain criteria in order to enter it.
@foreach (var serverRoom in Model) { if ((await AuthorizationService.AuthorizeAsync(User, serverRoom, "EnterServerRoom")).Succeeded) { <a asp-controller="Home" asp-action="ServerRoom” class="btn btn-primary">@serverRoom.Name</a> } }
Because we have a list of server rooms, we need to loop through each one, and perform a resource-based authorization check, using the instance of the ServerRoom class, along with the user, and the name of the policy. If the evaluation of the policy succeeds, the link is added to the page. Once you’ve gone through the process of setting up the policies for you application, adding Authorization checks to your View is very straight forward.
TagHelpers
The problem is, including authorization checks in your Razor pages can make the code cumbersome and difficult to read. For this reason, I suggest you utilize a tag helper when implementing authorization in a view. It’s not terribly difficult to implement your own tag helper, and it will save a ton of time. To start, lets look at what information our tag helper will need to perform our authorization checks.
• An instance of the AuthorizationService.
• The user from the HttpContext.
• The resource we are evaluating.
• The name of the policy that we will evaluate.
With these requirements in mind, lets look at how we can build our tag helper.
Start with a new class called AuthorizationTagHelper. We’re going to need to include a few libraries for our class, so we can reference instances of the AuthorizationService and HttpContextAccessor, as well as inherit from the TagHelper base class.
using Microsoft.AspNetCore.Authorization; using Microsoft.AspNetCore.Http; using Microsoft.AspNetCore.Razor.TagHelpers;
Now we need to define the class, and decorate it with an HtmlTargetElement attribute. This will allow us to specify the attribute in our Razor code that causes the TagHelper to modify the HTML. For this case, I’ve chosen to use “authorize”. The first two pieces of information we need to perform an evaluation are the IAuthorizationService and HttpContextAccessor. We can get those via dependency injection, so I’ll create private readonly variables for them, and add them to the constructor of our new class. The resource and name of the policy must be specified by the user, so I’ve created a property for each. Because the user will be assigning them in the view, I want to give each property an attribute name.
[HtmlTargetElement(Attributes = "authorize")] public class AuthorizationTagHelper : TagHelper { private readonly IAuthorizationService _authorizationService; private readonly IHttpContextAccessor _httpContextAccessor; [HtmlAttributeName("policy-name")] public string PolicyName { get; set; } [HtmlAttributeName("authorize-resource")] public object Resource { get; set; } public AuthorizationTagHelper(IAuthorizationService authorizationService, IHttpContextAccessor httpContextAccessor) { _authorizationService = authorizationService; _httpContextAccessor = httpContextAccessor; } }
The last thing we need to do is override the ProcessAsync function from the TagHelper base class.
public override async Task ProcessAsync(TagHelperContext context, TagHelperOutput output) { if (!(await _authorizationService.AuthorizeAsync(_httpContextAccessor.HttpContext.User, Resource, PolicyName)).Succeeded) { output.TagName = null; output.SuppressOutput(); } output.Attributes.RemoveAll("authorize"); }
Inside this method, we need to call AuthorizeAsync just like we would have done in the Razor code. The difference here is that we need to change the output based on the result of the authorization evaluation. If the output should not be included in the page, if the authorization check fails. To do this, we set the element the “authorize” tag was placed on to null to remove it from the page, and call output.SuppressOutput(). If the check succeeded, we don’t have to do anything. For sake of keeping the HTML clean, we can remove the “authorize” tag by calling output.Attributes.RemoveAll(“authorize”);
Here is an example of how our Tag Helper can clean up our Razor code.
<h3>Server Rooms:</h3> @foreach (var serverRoom in Model) { if ((await AuthorizationService.AuthorizeAsync(User, serverRoom, nameof(CustomPolicies.EnterServerRoom))).Succeeded) { <a asp-controller="Home" asp-action="ServerRoom" asp-route-id="@serverRoom.Name" class="btn btn-primary">@serverRoom.Name</a> } } <h3>TagHelperTest</h3> @foreach (var serverRoom in Model) { <a authorize policy-name=@nameof(CustomPolicies.EnterServerRoom) authorize-resource=serverRoom asp-controller="Home" asp-action="ServerRoom" asp-route-id="@serverRoom.Name" class="btn btn-primary">@serverRoom.Name</a> }
Looks much cleaner, and you won’t have to worry about traffic hitting those 403 errors!
Using JWTs with the ASP.NET UserManager
Last night I got stuck trying to fix a bug in a REST API that really drove me nuts, so I’ll share in the hopes it will save someone else a headache.
The service uses JWTs for authentication, and are issued in an authentication controller, while the data is provided by another controller. I have a CreateToken method that creates an instance of ApplicationUser
for the requested user, and adds some claims.
var claims = new[] { new Claim(JwtRegisteredClaimNames.Sub, user.UserName), new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()), new Claim(JwtRegisteredClaimNames.Email, user.Email) }.Union(userClaims);
I’m sure this looks familiar if you’ve used JWTs in .NET. The method creates a valid token, and returns it to the user. The user can then take that token and pass it along to a different controller for authentication. In my case, I had customized the ApplicationUser
to include a property called TenantID, and in the controller, I needed to create an instance of the ApplicationUser
, and use the TenantID.
var user = await _userManager.GetUserAsync(User);
The problem was that the GetUserAsync(User) always returned null. After screwing around with it for way too long, I went to GitHub to see what GetUserAsync was actually doing.
https://github.com/aspnet/Identity/blob/dev/src/Core/UserManager.cs
It calls GetUserID
, and passes in the ClaimsPrincipal
.
public virtual string GetUserId(ClaimsPrincipal principal) { if (principal == null) { throw new ArgumentNullException(nameof(principal)); } return principal.FindFirstValue(Options.ClaimsIdentity.UserIdClaimType); }
It wants to find the first claim associated with the UserIdClaimType….Which is what?
JwtRegisteredClaimNames.Sub
When I was creating the token, I was giving the Sub claim the user’s username, but ASP.NET identity was searching the database for the Id field in the AspNetUsers
table in SQL.
UGH.
So I changed the assignment when the token is created, and everything seems to work fine now.
var claims = new[] { new Claim(JwtRegisteredClaimNames.Sub, user.Id), new Claim(JwtRegisteredClaimNames.UniqueName, user.UserName), new Claim(JwtRegisteredClaimNames.Jti, Guid.NewGuid().ToString()), new Claim(JwtRegisteredClaimNames.Email, user.Email) }.Union(userClaims);
Going under the hood of ASP.NET Core 2 Authorization
Microsoft has made implementing simple Authorization requirements in ASP.NET Core 2 feel familiar to previous versions. But when you plan to implement something more complex, it’s very helpful to understand how things work under the hood. With that in mind, I’ll be writing a series of posts about how Authorization is currently implemented in Core 2.
Lay of The Land
It’s common for developers to think of Authorization and Authentication as the same thing, but they really have two different purposes. The job of Authentication is to confirm the identity of a user, while Authorization determines what a user can do. The criteria for allowing a user to perform an action may depend on the user’s membership in a role or group, a claim possessed by the user, or the state of a resource being acted upon. Microsoft’s design for Authorization in .NET Core 2 addresses those needs, and implements a separation of concerns, so that you can unit test authorization in your application. With this in mind, let’s begin with a brief introduction to policy based authorization.
What exactly is a Policy?
A policy is a list of requirements that a user must meet in order to perform an operation. Having a simple example to reference makes describing a policy easier, so I’ll use one from Microsoft’s “Blowdart” repository on Github.
services.AddAuthorization(options => { options.AddPolicy("AdministratorOnly", policy => policy.RequireRole("Administrator")); options.AddPolicy("EmployeeId", policy => policy.RequireClaim("EmployeeId", "123", "456")); });
In this case the first policy is named “AdministratorOnly”, and its requirement is that the user is assigned to the “Administrator” role. The second policy, “EmployeeId”, requires that the user has a claim called “EmployeeId” that has a value of 123 or 456. Each of these policies has a single requirement, although they could have more.
When you examine the source code for the AuthorizationPolicy
class, you’ll find a collection for the requirements. The requirements for a policy can range from requiring that the user is authenticated, is a member of a role, has a claim, or any number of other possibilities. The list of requirements is read only, so once that list is created, you can’t add more. Generally you’ll just create a new policy if needed.
There is another collection in the AuthorizationPolicy
class for AuthenticationSchemes
. They allow you to specify which authentication methods are allowed when a policy is being evaluated. This means you can have a policy that requires that the user is authenticated with a bearer token, or another that applies to cookies.
Policy.AuthenticationSchemes.Add(JwtBearerDefaults.AuthenticationScheme);
In the case of this policy, it would only be evaluated against an identity that was created with JWTBearer tokens, because the authentication scheme was added to the policy.
Looking back at the “Blowdart” example, the syntax for creating the policies might not be familiar to you. We’ll dig deeper later, but it’s suffice to say for now that IServiceCollection
extensions take in a dictionary of Action
delegates, and use the AuthorizationPolicyBuilder
to create the policies. The AuthorizationPolicyBuilder
class has various methods to add requirements to a policy and combine policies together. In the end, the Build method is called, and the new policy is created.
At this point, it probably looks like a big collection of strings, and you might be wondering where the actual work gets done. This brings us to authorization requirements.
What really goes into an authorization requirement?
In order to evaluate a requirement, you’ll need some information about the user, a chunk of code that does the evaluation, and sometimes metadata about the resource being secured. In ASP.NET Core, this is done with a few pieces working together. For the sake of type safety, there is an interface, IAuthorizationRequirement
, that has no methods or properties defined. If you browse through the source code, you’ll find that the predefined requirement classes implement this interface. But we still need a place to put that chunk of code that does the evaluation. That class is called the AuthorizationHandler
, and there are two types. One is used to evaluate requirements, and the second is designed to deal with requirements that secure a resource. For now, let’s set aside securing a resource.
public abstract class AuthorizationHandler : IAuthorizationHandler where TRequirement : IAuthorizationRequirement
The handler class is a generic that implements the HandleAsync
method. That method is provided with some context information, about the user, and in turn calls the HandleRequirementAsync
method.
protected abstract Task HandleRequirementAsync(AuthorizationHandlerContext context, TRequirement requirement);
The logic to evaluate the requirement is found in the HandleRequirementAsync
function. It takes in the needed information about the user in the form of the AuthorizationHandlerContext
class. The requirement will in all likelihood have some state, so it needs to be passed into the evaluation function too. Looking at an example will help make sense of it all. There is a NameAuthorizationRequirement
class that checks to make sure the name of the user is equal is one of the specified values.
protected override Task HandleRequirementAsync(AuthorizationHandlerContext context, NameAuthorizationRequirement requirement) { if (context.User != null) { if (context.User.Identities.Any(i => string.Equals(i.Name, requirement.RequiredName))) { context.Succeed(requirement); } } return Task.CompletedTask; }
Here you can see that the context has a property with the user information, and in this case the requirement has a string, RequiredName, that needs to match the name of the user. When they match, the Succeed method on the context is called. The context actually stores the results of all the requirement evaluations and aggregates the result.
One of the benefits of this design is that it can be unit tested. Here is one of Microsoft’s unit tests found on GitHub. It employs some classes I have not talked about yet, but you can see that it lets you set up a test, and validate that the requirements are evaluated properly. Separating the authorization logic from the business logic, makes this testing so much easier.
[Fact] public async Task CanRequireUserName() { // Arrange var authorizationService = BuildAuthorizationService(services => { services.AddAuthorization(options => { options.AddPolicy("Hao", policy => policy.RequireUserName("Hao")); }); }); var user = new ClaimsPrincipal( new ClaimsIdentity( new Claim[] { new Claim(ClaimTypes.Name, "Hao"), }, "AuthType") ); // Act var allowed = await authorizationService.AuthorizeAsync(user, "Hao"); // Assert Assert.True(allowed.Succeeded); }
I’ve presented a somewhat simplified view of requirements and how they are evaluated. It’s worth while to learn more about how resources can be included, and how logical operations can be used with a policy’s requirements for complex scenarios. More on that in the next post.
Securing an API with JSON Web Tokens (Configuring the API)
To add JWT bearer authentication to an ASP.net Core site, you’ll need to add the JWTBearer middleware to the application. The middleware is found in the Microsoft.AspNetCore.Authentication.JwtBearer namespace, and will allow the service to validate a token, and create a ClaimsPrincipal with the claims in the token. To add the middleware to the service, we’ll start by adding authentication with the options DefaultAuthenticationScheme and DefaultChallengeScheme set to JwtBearerDefaults.AuthenticationScheme. So when the [Authorize] attribute is put on controllers or actions, they will default to use JwtBearer authentication. In the Startup.cs file, there should be a ConfigureServices function that provides you with a IServiceCollection interface to do the configuring.
public void ConfigureServices(IServiceCollection services) { services.AddDbContext<RESTContext>(); services.AddIdentity<IdentityUser, IdentityRole>() .AddEntityFrameworkStores<RESTContext>(); services.AddAuthentication(options => { options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme; options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme; }) ... }
JwtBearer Options
The next service we need to add is the JwtBearer service, which allows configuration with the JwtBearerOptions parameter. Here are a list of several of the options.
- RequiredHttpsMetadata – JWT bearer tokens should never be passed around without using HTTPS. If someone were to intercept a token, they could use it to impersonate the owner of the token. Always set this option to true.
- AutomaticAuthenticate – When set to true, it will cause the service to automatically authenticate the user defined in the token.
- Audience – This value represents the service or resource that will receive the incoming token. Remember that list of registered claims that you can put in the payload of a token? Audience is one of them, and if the value in the token doesn’t match what is placed in this option, the token will be rejected.
- Authority – The authority option is used to specify the address of the authentication server that issued the token. If you set this option, the middleware will use it to get the public key that is used to validate the token’s signature. It will also verify that the issuer claim found in the token matches this value.
services.AddAuthentication(options => { options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme; options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme; }).AddJwtBearer(options => { options.Audience = Configuration["Tokens:Issuer"]; options.RequireHttpsMetadata = true;
Take note that I did not set the Authority option. The code sample is running a web application that issues and authenticates tokens that it creates. It is not reaching out to an authentication server to validate the token’s signature. If you were going to use a third party service to provide authentication, than this option becomes important.
Validation Options
We will also need to set the parameters that are used to validate a token. In ASP.net Core 2, this is done with the TokenValidationParameters object.
These are a list of the relevant options.
- ValidateIssuer(s) – Do you want to validate the issuer of the token? Generally this gets set to true.
- ValidIssuer – If you’re going to validate the issuer of the token, this is the value should match the issuer claim in the token.
- ValidateIssuerSigningKey – Do you want to validate the Issuer’s signing key?
- IssuerSigningKey – This is the public key used to validate JWT tokens. When you put the key here, it means that the service won’t need access to the token issuing authentication server. It could come from a file, certificate store, or as in the example below a configuration file. BTW don’t use the example below, it’s not your best security option.
services.AddAuthentication(options => { options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme; options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme; }).AddJwtBearer(options => { options.Audience = Configuration["Tokens:Issuer"]; options.RequireHttpsMetadata = true; options.TokenValidationParameters = new TokenValidationParameters { ValidateIssuerSigningKey = true, ValidateIssuer = true, ValidIssuer = Configuration["Tokens:Issuer"], IssuerSigningKey = new SymetricSecurityKey(Encoding.UTF9.GetBytes(Configuration["Tokens:Key"])), ValidateLifetime = true };
Now the last thing we need to do here is handle the event that gets fired when authentication has failed.
ValidateIssuer = true, ValidIssuer = Configuration["Tokens:Issuer"], IssuerSigningKey = new SymetricSecurityKey(Encoding.UTF9.GetBytes(Configuration["Tokens:Key"])), ValidateLifetime = true }; options.Events = new JwtBearerEvents() { OnAuthenticationFailed = c => { c.NoResult(); c.Response.StatusCode = 500; c.Response.ContentType = "text/plain"; return c.Response.WriteAsync("An error occurred processing your authentication."); } };
Applying the Attributes
Once the middleware is configured and added to the pipeline, you can use standard authorize attributes on your controllers.
You could also specify that a controller or action use JwtBearer authentication as part of the attribute if you need to be specific.
[Authorize(AuthenticationSchemess = JwtBearerDefaults.AuthenticationScheme)]