Erlang uid_server

Erlang uid_server is a 64bit integer unique id generation server inspired by Twitter snowflake. It can generate maximum 4096 unique ids per millisecond, 4 million ids per second. When it was tested in CentOS, it generated 4.3 billion ids in 16.4 hours, 75K ids per second per node.

Ids generated by this server has 64 bits, and it is time ordered

• 32 bits : unix seconds
• 10 bits : milli seconds
• 4 bits : group id(d.c. id)
• 6 bits : node id(machind id)
• 12 bits : serial number in milliseconds

To run uid_server, we highly recommend to run ntpd as a daemon, and in default, it is required to run it.

There are three files required to run this server

• configuration file (to override default configuration)
• nodes configuration file, which has host name, group id, and node id in it
• timestamp file, which is created automaticalluy when the server is running and it will be updated every 10 seconds

Default Configurations

[
    {requires_ntpd, true},                          % true or false, requires ntpd up and running for time sync.
    {nodes_config_file, "/etc/uid_nodes.conf"},     % server num configuration file
    {num_max_trial_new_unix_time, 128},             % numer of retry times when a clock goes backwards to re-generate an id before returns an error
    {timestamp_file, "/tmp/uid_timestamp.txt"}      % File path: the latest time in seconds. It will be updated in every 10 seconds, and used to check if system clock is forwarding
]  

System Clock Dependency

• When time drift happens it is tolerant up to 128 milliseconds by regenerating new id until it is greater than the last id.
• When the server is restarted, it looks up the last timestamp from the file and guarantees the new id is greater than that.

Compile

mkdir ebin
erlc -o ./ebin src/*.erl

Test

erlc -DTEST -o ./ebin src/*.erl
erl -noshell -pa ebin -eval 'eunit:test(uid_server,[verbose]).' -s init stop

Example

$ hostname=`hostname -s`; echo "[ {"$hostname",12,1} ]." > /tmp/uid_nodes.conf
$ echo  "[ {nodes_config_file, "/tmp/uid_nodes.conf"} ]." > /tmp/uid.conf
$ erl -pa ebin
1> uid_server:start("/tmp/uid.conf").
ok
2> uid_server:get().
5578856456111788032
3> uid_server:get().
5578856461279170560
4> uid_server:stop().
ok
5> eunit:test(uid_server,[verbose]).
======================== EUnit ========================
module 'uid_server'
    uid_server: setup_test...[0.003 s] ok
    uid_server: init_config_file_does_not_exist_test...ok
    uid_server: init_duplicate_hostname_in_nodes_config_test...[0.060 s] ok
    uid_server: init_duplicate_group_id_and_node_id_test...[0.049 s] ok
    uid_server: init_timestamp_initialization_test...[0.043 s] ok
    uid_server: init_invalid_timestamp_test...[0.053 s] ok
    uid_server: init_valid_timestamp_test...[0.050 s] ok
    uid_server: get_unique_id_with_future_clock_100ms_test...ok
    uid_server: get_unique_id_with_future_clock_an_hour_test...[0.258 s] ok
    uid_server: valid_id_generation_test_with_server_up_and_running_test...[0.042 s] ok
    uid_server: forwarding_id_generation_test_with_server_up_and_running_test...    200000 ids in 2 seconds, 73520/sec
[2.961 s] ok
    uid_server: teardown_test...ok
    [done in 3.553 s]
=======================================================
    All 12 tests passed.
ok