Introduction
If you use a GeoIP database, you’re probably familiar with MaxMind’s MMDB format.
At MaxMind, we created the MMDB format because we needed a format that was very fast and highly portable. MMDB comes with supported readers in many languages. In this blog post, we’ll create an MMDB file which contains an access list of IP addresses. This kind of database could be used when allowing access to a VPN or a hosted application.
Prerequisites
The code samples in this post use the
Go mmdbwriter module to
create MMDB files and the
maxminddb-golang module
to read them. You can also read MMDB files with the officially supported
.NET,
Java,
Node.js,
PHP,
Python, and
Ruby readers, in addition
to unsupported third party MMDB readers. Many are listed on the
GeoIP download page. So, as far
as deployments go, you’re not constrained to any one language when you want to
read from the database.
You will need:
- Go 1.24 or later installed, with
goin your$PATH - the
mmdbinspecttool installed and in your$PATH - a basic understanding of Go and of IP addresses and CIDR notation
Getting Started
In our example, we want to create an access list of some IP addresses to allow them access to a VPN or a hosted application. For each IP address or IP range, we need to track a few things about the person who is connecting from this IP.
- name
- development environments to which they need access
- an arbitrary session expiration time, defined in seconds
To do so, we create the following Go program:
1package main
2
3import (
4 "fmt"
5 "log"
6 "net"
7 "os"
8
9 "github.com/maxmind/mmdbwriter"
10 "github.com/maxmind/mmdbwriter/mmdbtype"
11)
12
13func main() {
14 // Create a new MMDB tree.
15 writer, err := mmdbwriter.New(mmdbwriter.Options{
16 // "DatabaseType" is some arbitrary string describing the database.
17 // At MaxMind we use strings like "GeoIP2-City", "GeoIP2-Country", etc.
18 DatabaseType: "My-IP-Data",
19
20 // "Description" is a map where the keys are language codes and the
21 // values are descriptions of the database in that language.
22 Description: map[string]string{
23 "en": "My database of IP data",
24 "fr": "Mon Data d'IP",
25 },
26
27 // "IPVersion" can be either 4 or 6.
28 IPVersion: 4,
29
30 // "RecordSize" is the record size in bits. Either 24, 28, or 32.
31 RecordSize: 24,
32 })
33 if err != nil {
34 log.Fatal(err)
35 }
36
37 // Define employee data to insert.
38 employees := map[string]mmdbtype.Map{
39 // Jane connects from a single IP address.
40 "214.71.225.36/32": {
41 "environments": mmdbtype.Slice{
42 mmdbtype.String("development"),
43 mmdbtype.String("staging"),
44 mmdbtype.String("production"),
45 },
46 "expires": mmdbtype.Uint32(86400),
47 "name": mmdbtype.String("Jane"),
48 },
49 // Klaus could connect from any of 16 IP addresses (/28).
50 "6.248.221.67/28": {
51 "environments": mmdbtype.Slice{
52 mmdbtype.String("development"),
53 mmdbtype.String("staging"),
54 },
55 "expires": mmdbtype.Uint32(3600),
56 "name": mmdbtype.String("Klaus"),
57 },
58 }
59
60 for cidr, data := range employees {
61 _, network, err := net.ParseCIDR(cidr)
62 if err != nil {
63 log.Fatal(err)
64 }
65 if err := writer.Insert(network, data); err != nil {
66 log.Fatal(err)
67 }
68 }
69
70 // Write the database to disk.
71 fh, err := os.Create("users.mmdb")
72 if err != nil {
73 log.Fatal(err)
74 }
75 defer fh.Close()
76
77 _, err = writer.WriteTo(fh)
78 if err != nil {
79 log.Fatal(err)
80 }
81
82 if err := fh.Close(); err != nil {
83 log.Fatal(err)
84 }
85
86 fmt.Println("users.mmdb has now been created")
87}
The Code in Review
Step 1
Create a new
mmdbwriter.Tree by
calling
mmdbwriter.New(). The
tree is where the database is stored in memory as it is created.
1writer, err := mmdbwriter.New(mmdbwriter.Options{...})
The options we’ve used are all commented in the script, but there are additional options available. To keep things simple (and easily readable), we used IPv4 to store addresses in this example, but you could also use IPv6.
Step 2
For each IP address or range, define the data using
mmdbtype types
and call the
Insert()
method. This method takes a *net.IPNet (from
net.ParseCIDR()) and an
mmdbtype.DataType value.
1for cidr, data := range employees {
2 _, network, err := net.ParseCIDR(cidr)
3 if err != nil {
4 log.Fatal(err)
5 }
6 if err := writer.Insert(network, data); err != nil {
7 log.Fatal(err)
8 }
9}
The MMDB format is strongly typed. You must define your data using mmdbtype
types such as Map, String, Uint32, and Slice. You’re encouraged to
review
the full list of available types.
We’ve inserted information about two employees, Jane and Klaus. They’re both on different IP ranges. You’ll see that Jane has access to more environments than Klaus has, but Klaus could theoretically connect from any of 16 different IP addresses (/28) whereas Jane will only connect from one (/32).
Step 3
Open a file and write the database to disk.
1fh, err := os.Create("users.mmdb")
2// ...
3_, err = writer.WriteTo(fh)
Let’s Do This
First, initialize a Go module and then run the script:
1$ go mod init mmdb-tutorial
2$ go mod tidy
3$ go run main.go
4users.mmdb has now been created
You should also see the users.mmdb file in the folder from which you ran the
script.
Reading the File
Now we have our brand new MMDB file. Let’s read the information we stored in it.
Save this as main.go (replacing the previous one) and run go mod tidy to
fetch the new dependency.
1package main
2
3import (
4 "encoding/json"
5 "fmt"
6 "log"
7 "net/netip"
8 "os"
9
10 "github.com/oschwald/maxminddb-golang/v2"
11)
12
13func main() {
14 if len(os.Args) < 2 {
15 log.Fatal("Usage: go run main.go <ip_address>")
16 }
17 ipStr := os.Args[1]
18
19 db, err := maxminddb.Open("users.mmdb")
20 if err != nil {
21 log.Fatal(err)
22 }
23 defer db.Close()
24
25 fmt.Printf("Description: %s\n", db.Metadata.Description["en"])
26
27 ip, err := netip.ParseAddr(ipStr)
28 if err != nil {
29 log.Fatalf("Invalid IP address: %s", ipStr)
30 }
31
32 result := db.Lookup(ip)
33 if err := result.Err(); err != nil {
34 log.Fatal(err)
35 }
36 if !result.Found() {
37 fmt.Println("No record found for", ipStr)
38 return
39 }
40
41 var record map[string]any
42 if err := result.Decode(&record); err != nil {
43 log.Fatal(err)
44 }
45
46 out, err := json.MarshalIndent(record, "", " ")
47 if err != nil {
48 log.Fatal(err)
49 }
50 fmt.Println(string(out))
51}
Reading the File: Review
Step 1
Ensure that the user has provided an IP address via the command line.
1if len(os.Args) < 2 {
2 log.Fatal("Usage: go run main.go <ip_address>")
3}
Step 2
We create a new
maxminddb.Reader
by calling
maxminddb.Open(),
using the name of the file we just created as the argument.
1db, err := maxminddb.Open("users.mmdb")
Step 3
Check the metadata. This is optional, but here we print the description we added to the metadata in the previous script.
1fmt.Printf("Description: %s\n", db.Metadata.Description["en"])
db.Metadata is a
Metadata
struct. Beyond the Description, it provides extensive details about the
generated file.
Step 4
We perform a record lookup and print it as formatted JSON. The Lookup method
returns a
Result.
We check for errors with Err(), then use Found() to determine whether the IP
has a record, and finally Decode the result.
1result := db.Lookup(ip)
2if err := result.Err(); err != nil {
3 log.Fatal(err)
4}
5if !result.Found() {
6 fmt.Println("No record found for", ipStr)
7 return
8}
9var record map[string]any
10if err := result.Decode(&record); err != nil {
11 log.Fatal(err)
12}
You could also define a struct with maxminddb tags for type-safe decoding:
1type UserRecord struct {
2 Name string `maxminddb:"name"`
3 Environments []string `maxminddb:"environments"`
4 Expires uint32 `maxminddb:"expires"`
5}
Running the Script
Now let’s run the script and perform a lookup on Jane’s IP address:
1$ go run main.go 214.71.225.36
2Description: My database of IP data
3{
4 "environments": [
5 "development",
6 "staging",
7 "production"
8 ],
9 "expires": 86400,
10 "name": "Jane"
11}
We see that our Description and our map of user data is returned exactly as we
initially provided it. But what about Klaus, is he also in the database?
1$ go run main.go 6.248.221.64
2Description: My database of IP data
3{
4 "environments": [
5 "development",
6 "staging"
7 ],
8 "expires": 3600,
9 "name": "Klaus"
10}
11
12$ go run main.go 6.248.221.79
13Description: My database of IP data
14{
15 "environments": [
16 "development",
17 "staging"
18 ],
19 "expires": 3600,
20 "name": "Klaus"
21}
22
23$ go run main.go 6.248.221.80
24Description: My database of IP data
25No record found for 6.248.221.80
We gave Klaus an IP range of 6.248.221.67/28, which translates to
6.248.221.64 to 6.248.221.79. You can see that when we get to 6.248.221.80
there is no record at this address.
Iterating Over the Search Tree
It takes time to look up every address individually. Is there a way to speed things up? As it happens, there is.
1package main
2
3import (
4 "encoding/json"
5 "fmt"
6 "log"
7
8 "github.com/oschwald/maxminddb-golang/v2"
9)
10
11func main() {
12 db, err := maxminddb.Open("users.mmdb")
13 if err != nil {
14 log.Fatal(err)
15 }
16 defer db.Close()
17
18 for result := range db.Networks() {
19 var record map[string]any
20 if err := result.Decode(&record); err != nil {
21 log.Fatal(err)
22 }
23 if len(record) == 0 {
24 continue
25 }
26 out, err := json.MarshalIndent(record, "", " ")
27 if err != nil {
28 log.Fatal(err)
29 }
30 fmt.Printf("%s\n%s\n\n", result.Prefix(), string(out))
31 }
32}
Iterating: Review
Step 1
As in the previous example, we open the database with maxminddb.Open().
Step 2
To iterate over every network in the database, we use the
Networks()
method. This returns a Go range function
iterator. Each iteration yields a
Result
containing the network prefix and its associated record.
By default, Networks() skips aliased IPv4 networks in IPv6 databases and
networks without data.
Let’s look at the output.
1$ go run main.go
26.248.221.64/28
3{
4 "environments": [
5 "development",
6 "staging"
7 ],
8 "expires": 3600,
9 "name": "Klaus"
10}
11
12214.71.225.36/32
13{
14 "environments": [
15 "development",
16 "staging",
17 "production"
18 ],
19 "expires": 86400,
20 "name": "Jane"
21}
The output shows each network and its associated record. Note that even though
we specified Klaus’s range as 6.248.221.67/28, the writer correctly stored it
as 6.248.221.64/28, the canonical form of the network.
The Mashup
To extend our example, let’s take the data from an existing GeoIP database and combine it with our custom employee data. We’ll start with the GeoLite City database and enrich it with our access list information.
You may need to download the GeoLite City database or use geoipupdate to obtain it.
Save this as main.go (replacing the previous one) and run go mod tidy to
fetch the new dependencies.
We load the existing database using
mmdbwriter.Load(),
then merge our employee data into the matching IP ranges using
InsertFunc()
with the
TopLevelMergeWith
inserter. This adds our new top-level keys to the existing GeoLite records
without overwriting any of the original data.
1package main
2
3import (
4 "fmt"
5 "log"
6 "net"
7 "os"
8
9 "github.com/maxmind/mmdbwriter"
10 "github.com/maxmind/mmdbwriter/inserter"
11 "github.com/maxmind/mmdbwriter/mmdbtype"
12)
13
14func main() {
15 // Load the existing database that we want to enrich.
16 writer, err := mmdbwriter.Load("GeoLite2-City.mmdb", mmdbwriter.Options{})
17 if err != nil {
18 log.Fatal(err)
19 }
20
21 // Define employee data to merge into the existing records.
22 employees := map[string]mmdbtype.Map{
23 "214.71.225.36/32": {
24 "environments": mmdbtype.Slice{
25 mmdbtype.String("development"),
26 mmdbtype.String("staging"),
27 mmdbtype.String("production"),
28 },
29 "expires": mmdbtype.Uint32(86400),
30 "name": mmdbtype.String("Jane"),
31 },
32 "6.248.221.67/28": {
33 "environments": mmdbtype.Slice{
34 mmdbtype.String("development"),
35 mmdbtype.String("staging"),
36 },
37 "expires": mmdbtype.Uint32(3600),
38 "name": mmdbtype.String("Klaus"),
39 },
40 }
41
42 for cidr, data := range employees {
43 _, network, err := net.ParseCIDR(cidr)
44 if err != nil {
45 log.Fatal(err)
46 }
47 // InsertFunc with TopLevelMergeWith merges our new top-level keys
48 // into the existing GeoLite record, rather than replacing it.
49 if err := writer.InsertFunc(network, inserter.TopLevelMergeWith(data)); err != nil {
50 log.Fatal(err)
51 }
52 }
53
54 // Write the enriched database to disk.
55 fh, err := os.Create("users-enriched.mmdb")
56 if err != nil {
57 log.Fatal(err)
58 }
59 defer fh.Close()
60
61 _, err = writer.WriteTo(fh)
62 if err != nil {
63 log.Fatal(err)
64 }
65
66 if err := fh.Close(); err != nil {
67 log.Fatal(err)
68 }
69
70 fmt.Println("users-enriched.mmdb has now been created")
71}
Now, when we look up our employee IP addresses in the enriched database using
mmdbinspect -db users-enriched.mmdb 214.71.225.36, we can see that the records
contain both the geographic data and our custom fields. mmdbinspect outputs
YAML by default:
1database_path: users-enriched.mmdb
2requested_lookup: 214.71.225.36
3network: 214.71.225.36/32
4record:
5 continent:
6 code: NA
7 geoname_id: 6255149
8 names:
9 de: Nordamerika
10 en: North America
11 ...
12 country:
13 geoname_id: 6252001
14 iso_code: US
15 names:
16 en: United States
17 ...
18 environments:
19 - development
20 - staging
21 - production
22 expires: 86400
23 location:
24 accuracy_radius: 1000
25 latitude: 37.751
26 longitude: -97.822
27 time_zone: America/Chicago
28 name: Jane
29 registered_country:
30 ...
The record now contains both the original geographic fields (continent,
country, location) and our custom access list fields (environments,
expires, name).
The Mashup: Review
To enrich the existing database, we make two key changes from our original “Getting Started” script:
Step 1
Instead of creating a new tree with mmdbwriter.New(), we load the existing
database:
1writer, err := mmdbwriter.Load("GeoLite2-City.mmdb", mmdbwriter.Options{})
This loads the entire GeoLite database into a writable tree that we can modify.
Step 2
Instead of Insert(), we use InsertFunc() with
inserter.TopLevelMergeWith():
1if err := writer.InsertFunc(network, inserter.TopLevelMergeWith(data)); err != nil {
2 log.Fatal(err)
3}
The TopLevelMergeWith inserter merges our new map keys into the existing
record’s map. If we used Insert() instead, the default behavior would replace
the existing GeoLite record entirely, losing all the geographic data.
For a more detailed walkthrough of this pattern, see our article on enriching MMDB files with your own data using Go.
Deploying Our Application
Now we’re at the point where we can make use of our database. With just a few lines of code you can now use your MMDB file to assist in the authorization of your application or VPN users. For example, you might include the following in your authentication handler:
1import (
2 "log"
3 "net/netip"
4
5 "github.com/oschwald/maxminddb-golang/v2"
6)
7
8type UserRecord struct {
9 // Use a pointer so we can distinguish "field absent" from "empty string".
10 Name *string `maxminddb:"name"`
11 Environments []string `maxminddb:"environments"`
12 Expires uint32 `maxminddb:"expires"`
13 Location struct {
14 TimeZone string `maxminddb:"time_zone"`
15 } `maxminddb:"location"`
16}
17
18func main() {
19 // Open the reader once at startup and reuse it. It is safe for
20 // concurrent use.
21 db, err := maxminddb.Open("/path/to/users-enriched.mmdb")
22 if err != nil {
23 log.Fatal(err)
24 }
25 defer db.Close()
26
27 // Pass db to your HTTP handlers, auth middleware, etc.
28 // ...
29}
30
31func isIPValid(db *maxminddb.Reader, ipStr string) (*UserRecord, bool) {
32 ip, err := netip.ParseAddr(ipStr)
33 if err != nil {
34 return nil, false
35 }
36
37 var record UserRecord
38 if err := db.Lookup(ip).Decode(&record); err != nil {
39 log.Printf("MMDB lookup error for %s: %v", ipStr, err)
40 return nil, false
41 }
42
43 // A nil Name means this IP has no employee data. It may exist in
44 // GeoLite but is not on our access list.
45 if record.Name == nil {
46 return nil, false
47 }
48
49 // Use record.Expires to set session expiration.
50 // Use record.Location.TimeZone for displaying dates and times.
51 return &record, true
52}
Here’s a quick summary of what’s going on:
- As part of your deployment you’ll naturally need to include your MMDB file, stored in the location of your choice.
- You’ll need to create a
maxminddb.Readerby callingmaxminddb.Open(). Open it once and reuse it. The reader is safe for concurrent use. - If
Nameis nil, the IP exists in GeoLite but has no employee data, so it is not on our access list. - If the IP is found, you can set a session expiration.
- If the IP is found, you can also use
record.Location.TimeZonefrom the GeoLite data to customize the user’s experience. Keep in mind that this field comes from the GeoLite database and may not be available for all IP addresses.
Pro Tips
Merge Strategies
The mmdbwriter module provides several
inserter functions
that control what happens when you insert data for a network that already has a
record:
ReplaceWith: the default when usingInsert(). The new data completely replaces the existing record.TopLevelMergeWith: merges top-level map keys. Existing keys that aren’t in the new data are preserved.DeepMergeWith: recursively merges nested maps and slices.
Choose the strategy that fits your use case. For enriching existing databases,
TopLevelMergeWith is usually what you want.
Thread Safety
The Insert and InsertFunc methods are not safe for concurrent use. Perform
all insertions from a single goroutine before calling WriteTo.
Taking This Further
Today we’ve shown how you can create your own MMDB database and enrich it with data from a GeoLite database. We’ve only included a few data points, but MaxMind databases contain much more data you can use to build a solution to meet your business requirements.
For a more detailed walkthrough of enriching MMDB files, see our article on
enriching MMDB files with your own data using
Go.
For full API documentation, see the
mmdbwriter package documentation.
