Geo Enrichment

MinusOneDB can append geographic information to records at write time, derived from the IP address the write arrived from. Nothing in your payload has to carry location: enable the feature, add the geo properties to your schema, and the _m1.geo.* fields are populated for you as records are published.

This is IP-derived location. It is accurate enough to answer "which state are my users in" and not accurate enough to answer "which building". Every record carries _m1.geo.accuracy_radius, the radius in kilometres within which the true location is likely to fall — read it before drawing conclusions from a coordinate.

Enabling geo

Geo enrichment is switched on with the geo system parameter, and the address it reads is controlled by property-ip. Enrichment applies to items written to the archive layer and to session stores.

In an environment template it goes in the configure block alongside your other system settings:

"configure": [{
    "system": {
        "geo": "true",
        "publish-permissive": "true"
    }
}]

Adding the geo properties

Enabling the setting does not create the fields. An environment with geo turned on must carry the geo properties in its schema, and the enrichment populates whichever of them exist. This page enumerates that set.

The property names below are the defaults. They can be mapped to alternate names if they clash with your own naming — the field reference is a description of the data available, not a fixed set of identifiers.

A ready-made property file ships with the event capture kits as schema/geo.json, so the usual path is to point the schema endpoint at that file rather than write the definitions by hand:

{ "schema/add": { "properties": "@../schema/geo.json" } }

Two supporting properties are included in that file and are worth keeping. The enrichment depends on the first, and most geo queries want the second:

Field	Type	Description
_m1.ip	string	The IP address of the user. This is the input the geo data is derived from.
_m1.receivedUTC	date	The date of the request.

Geo field reference

Position

Field	Type	Description
_m1.geo.position	latlong	The approximate latitude and longitude of the location associated with the network.
_m1.geo.latitude	double	The approximate latitude of the location associated with the network.
_m1.geo.longitude	double	The approximate longitude of the location associated with the network.
_m1.geo.accuracy_radius	integer	The radius in kilometers around the specified location where the IP address is likely to be.

Place

Field	Type	Description
_m1.geo.continent_code	string	The continent code for this location.
_m1.geo.continent_name	string	The continent name for this location.
_m1.geo.country_iso_code	string	The country code for the country associated with the location.
_m1.geo.country_name	string	The country name for this location in the specific locale.
_m1.geo.subdivision_1_iso_code	string	The region-portion of the code for the first level region associated with the IP address.
_m1.geo.subdivision_1_name	string	The first level subdivision name for this location in the specific locale.
_m1.geo.subdivision_2_iso_code	string	The region-portion of the code for the second level region associated with the IP address.
_m1.geo.subdivision_2_name	string	The second level subdivision name for this location.
_m1.geo.city_name	string	The city name for this location.
_m1.geo.postal_code	string	A postal code close to the user's location.
_m1.geo.metro_code	string	The metro code associated with the IP address.
_m1.geo.time_zone	string	The time zone associated with location.

Identifiers

Field	Type	Description
_m1.geo.geoname_id	integer	A unique identifier for the network's location as specified by GeoNames.
_m1.geo.registered_country_geoname_id	integer	The country in which the ISP has registered the network.
_m1.geo.represented_country_geoname_id	integer	The country which is represented by users of the IP address.
_m1.geo.network	string	The IPv4 or IPv6 network.
_m1.geo.locale_code	string	The locale that the names in this row are in.

Flags

Field	Type	Description
_m1.geo.is_anonymous_proxy	boolean	Indicates if the IP address is used by an anonymizing service.
_m1.geo.is_in_european_union	boolean	Indicates if the country associated with the location is a member state of the European Union.
_m1.geo.is_satellite_provider	boolean	Indicates if the IP address is a satellite provider.

The latlong type

_m1.geo.position uses the latlong property type, which holds a latitude and longitude together as a single coordinate rather than as two independent numbers.

latlong is a valid value for type when adding properties, and may be used for your own properties as well as for the geo fields:

[{ "name": "storeLocation", "type": "latlong",
   "description": "Coordinates of the retail location." }]

The individual _m1.geo.latitude and _m1.geo.longitude values are also stored as double properties, so a coordinate is available both as a pair and as two separately queryable numbers.

Distance queries are not yet documented

Filtering or sorting by distance from a point is outside what this page covers. Until that is written up, treat latlong as a storage type: it holds the coordinate reliably, and range queries against the separate latitude and longitude doubles are the approach we can currently document.

Querying geo data

Geo fields behave like any other property once populated. They facet, filter and sort the way their underlying types do.

Count events by state:

curl

curl https://test-m1.minusonedb.com/query \
-d 'store=index&q=*&json={
  "facet": {
    "by_state": {
      "field": "_m1.geo.subdivision_1_iso_code",
      "type": "terms"
    }
  }
}' \
-H "m1-auth-token: $myToken"

m1 client

m1 test-m1 query -store index -q '*' -json '{
  "facet": {
    "by_state": {
      "field": "_m1.geo.subdivision_1_iso_code",
      "type": "terms"
    }
  }
}'

Restrict a query to one country, then facet by city:

m1 test-m1 query -store index -q '_m1.geo.country_iso_code:US' -json '{
  "facet": {
    "by_city": {
      "field": "_m1.geo.city_name",
      "type": "terms"
    }
  }
}'

Bound a query to a rectangle using the latitude and longitude doubles:

m1 test-m1 query -store index \
  -q '_m1.geo.latitude:[40.4 TO 40.9] AND _m1.geo.longitude:[-74.3 TO -73.7]'

Notes

• Enrichment happens at write time. Records published before geo was enabled, or before the properties existed, are not backfilled.
• Records whose IP cannot be matched are written without geo values rather than being rejected. Do not assume every record has a position.
• _m1.geo.is_anonymous_proxy and _m1.geo.is_satellite_provider flag addresses whose apparent location is unlikely to reflect the user. Exclude them before reporting on geography.