Few weeks ago, Troy Hunt has released password hash dumps from haveibeenpwned.com site. Dumps are large, splitted to 3 parts and contains 324+ millions of hashes. In this blog post I will show you how to integrate that large hash dump with Microsoft Active Directory and enable DC servers to check against that list before allowing user to change their password.
Microsoft has one feature that has been present since Windows server 2003 and it’s called password filters. It’s not often used as it’s meant to be used as an additional method for adding more complexity to password requirements in larger organisations. The smaller organisations and companies are sticking with the rules that are already present in Windows (both server and workstations), which are:
- enforce password history
- minimum password age
- maximum password age
- minimum password length
- password must meet complexity requirements
- store passwords using reverse encryption
There are some commercial solutions that can add more complex requirements to this list, but price tag is quite high. As soon as you see “contact us for price” you can count with that.
When Troy released hashes I got idea to implement them in some way with AD (Active Directory) to enable DCs (Domain Controllers) to verify passwords against it. In past few months Nist and Microsoft have came out with the new password guidelines as well, but I won’t write about that here, if you are interested to read about it you can read it on following links:
- Troy Hunt – Passwords Evolved: Authentication Guidance for the Modern Era
- Microsoft Azure Password Policy
- NIST Digital Identity Guidelines, SP 800-63-3, June 2017
In short or tl;dr new guidelines are recommending removing password complexity, history and aging requirements as they are not adding to password security at all and are recommending comparing passwords or hashes to dictionary lists so that easy passwords can be eliminated on time as well as keeping length of passwords at minimum of 8 characters.
So today I will write about comparing passwords against hashes. While looking how to do this in proper way I stumbled upon OpenPasswordFilter by Josh Stone on Github. Code was easy to understand and it was really easy to start extending, in my case to support database validation.
OpenPasswordFilter in its current form is doing validation against 2 password lists and it’s doing partial and full-word validation of passwords. It is based on 2 components, one is OpenPasswordFilter.dll file which is integrating into AD, the other one is OPFService (Windows service) which is listening on loopback address for client connections. Client (in this case .dll) is connecting to service, sending first “hello” sequence and then password after that. If service has found password as a partial or full match it’s returning boolean which .dll needs as answer. Password filters are working in a way that all of them (you can have as many as you want) have to return true, if any of them fails password change is denied by DC.
I have extended OpenPasswordFilter and added following to it:
- SQL server hash validation
- Logging of exceptions
- Custom “hello” keyword so you can change it to whatever you like from config file
My fork of it is available here on Github, you will find more details of it there, feel free to comment or create an issue if needed.
So to get OpenPasswordFilter or OPF working (my way) we need to do following:
- install SQL server & SQL Management Studio
- create database, in this example called PwnedPwdDB
- create table BadHashes
- create unique index for sorting and elimination of duplicate hashes
- create Hashes view for easier data loading with bcp
- download data from haveibeenpwned.com and unzip it to some directory
- import to SQL server with bcp
- test for hashes
- install OPF
I won’t explain installation of SQL server, you can download SQL Server Express if you don’t have one running somewhere. Install is quite straightforward. After installation, create PwnedPwdDB database and run following code to create table, view and index on it.
Index is needed for faster searching of data, after all we are searching trough 324+ million of records. View is there to make it easier to load data with bcp as loading data directly to BadHashes table requires us to have an hash_id as well, that’s why we are loading data to the table trough view we created.
For loading of data to database I’m using bcp (bulk copy), Microsoft’s util for this kind of situations, loading of data to/from SQL server when data is formatted in special way, in our case with new line.
Now we are ready to import data. As data is quite big and bcp is sending 1000 rows per insert, here is the command I have used to load 10 000 rows per insert which was quite OK value for my server without too much deadlocking on the it.
bcp dbo.Hashes IN pwned-passwords-1.0.txt -T -S Server_ip\instance_name -d PwnedPwdDB -c -b 10000
Repeat command for other 2 files, pwned-passwords-update-1.txt and pwned-passwords-update-2.txt and any other you might have with SHA1 hashes.
In case that import fails or you get some client error, you can just repeat the commands. Thanks to that index above, if record has been found in database it won’t be imported again, it will be reported as a warning from bcp and import will continue on.
After import is done, and it will take some time based on performance of your SQL server, we are ready to test it. To simple test hash against database run this command, replace password with password you want to test. If you get hash back from database it means it has been compromised on some site breach and should not be used.
So after you have loaded data, tested it it’s time to download and install OPFService.
Head to OpenPasswordFilter fork that I did and download and compile source code or download precompiled version. I recommend that you compile your own version just because these are very sensitive things we are working on. If you decide to download precompiled versions do check hash values on .zip files to be sure they are from me and those I committed in a latest commit to repository.
Installation is quite simple:
- When you have release (compiled or downloaded) folder just move it to a DC server, start elevated command prompt and change to that directory and write following command to install service
- for 64bit Windows Server:
- for 32bit Windows Server:
- for 64bit Windows Server:
- change settings in OPFService.config to match your database settings
- start service from services.msc or by writing
sc start OPFin the same command prompt
- if service has been started you can test it by typing
OPFTest passwordand you should get following output
This means that password has been found and service is returning failure which will later on tell to DC not to accept password change. If you try any other more secure password that is not compromised you should get success as response which means that password is valid and not compromised.
Now the last thing remaining to do is to copy/move OpenPasswordFilter.dll to c:\Windows\System32 directory and validate registry key
HKLM\SYSTEM\CurrentControlSet\Control\Lsa\Notification Packages with regedit that it contains OpenPasswordFilter key as well. If all of that is set you just need to restart DC and test password change from normal Windows UI, ctrl+alt+del -> change password is the simplest way.
Note: if you have more than one DC you will have to install OPFService on all of them, easiest way is to just copy release directory from the first server to another one and run service installation command.
- Troy Hunt on Twitter, follow him for loads of great security info, security courses (free and paid) and lot of useful info
- haveIbeenpwned.com – if you are not subscribed there (and you should be), subscribe all of your email addresses to get informed if breach happens on some site where you are registered
- bcp documentation
- OpenPasswordFilter – original branch by Josh Stone
- OpenPasswordFilter with SQL support – my fork of OPF
- Microsoft Password Filter installation docs
In case of any questions or problems leave me a comment and I will get back to you.
As Mitch pointed out in comments, you will have to install .Net 3.5 (or higher) to be able to start service properly as well as add key
HKLM\SYSTEM\CurrentControlSet\Control\Lsa\Notification Packages if it doesn’t exists.