D2: The Libraries Behind WORX, Now on NuGet and npm
Motivations
Throughout my career I have been unable to share my work with others. As with most things wrong in my life, this is yet another example of a 3-letter agency keeping their boot on my neck. In this case, that agency is the NDA. Jokes aside, as I have built progressively cooler things, I have wanted to share them but never found a socially or contractually appropriate medium. With my recent work on WORX, that changes. Originally, I hosted the code here, under a noncommercial license, but, as progress has continued on the product, the thing that really motivated me to fully open source a good portion of it (under Apache-2.0) is wanting to do the exact opposite. My reactionary fear was that by taking down the only public example of any (non-front-end) source code I've ever written, I'd be robbing myself of an ez W in the eyes of anyone who actually looks at stuff like this (for whatever reason). So, instead, I essentially split the repo in two and the D2-Public repo is a projection of a subset of my private one that contains spooky trade secrets.
Disclaimer
While these libraries are relatively opinionated (like all good software engineers), I have created things that are genuinely purpose-built to be widely re-used in my future projects. With that being said, this is just a mirror and some of the design decisions made are 1000% a reflection of my own constraints and needs. In addition, WORX is still a WIP so everything is subject to change. Regardless, you will find some useful libraries here if you're looking for something that just works and won't get rugpulled the second it gets 100K MAU with a new dual-tier random/commercial license.
Highlights
Not to be confused with a popular song by Ye (A.K.A Kanye West), here are some of the highlights of my repo (all packages are published on NuGet and npm):
#1 Geo(graphy)
My geo package contains everything you need to lay the groundwork for geography data in your project including countries, their subdivisions, currencies, locales, time zones and geopolitical entities. Instead of this data being stored in your database, it stays in memory and has plenty of paths for O(1) lookups.
Here are some lookup patterns from the library, in both TypeScript and C# (differs where applicable):
Countries.USgets a country by a typed ISO 3166-1 Alpha-2 Codecountry.Subdivisionsgets all the subdivisions for a country (wherecountryis aCountry)country.GeopoliticalEntityShortCodes.Contains(GeopoliticalEntityCode.EU)(C#)country.geopoliticalEntityShortCodes.has(GeopoliticalEntityCode.EU)(TS) O(1) check if a country is in a given geopolitical entity (by its code)SubdivisionLookup.ByCode[Subdivisions.US.NY](C#)SubdivisionLookup.byCode[Subdivisions.US.NY](TS) get a subdivision by its typed ISO short code.Subdivisions.US.NYgives you the code "US-NY"
There's more, you get the idea.
On the TS side, there are per-catalog subpath exports so bundlers can tree-shake what they need.
To give you an idea on the level of detail each catalog has, here are the properties on some:
Country:
Iso31661Alpha2Code, Iso31661Alpha3Code, Iso31661NumericCode, DisplayName, OfficialName, EndonymDisplayName, EndonymOfficialName, PhoneNumberPrefix, PhoneNumberNationalFormat, PhoneNumberMinDigits, PhoneNumberMaxDigits, FirstDayOfWeek, WeekendStart, WeekendEnd, MeasurementSystem, PrimaryLanguageIso6391Code, PrimaryLanguage, PrimaryCurrencyIso4217AlphaCode, PrimaryCurrency, PrimaryLocaleIetfBcp47Tag, PrimaryLocale, SovereignCountryIso31661Alpha2Code, SovereignCountry (what country owns this country), TerritoryIso31661Alpha2Codes, Territories (what countries this country owns), SubdivisionIso31662Codes, Subdivisions (a country's states, provinces or regions), LocaleIetfBcp47Tags, Locales, GeopoliticalEntityShortCodes, GeopoliticalEntities, CurrencyIso4217AlphaCodes, Currencies, Deprecation
Subdivision:
Iso31662Code, ShortCode (TX for example), DisplayName, OfficialName, EndonymDisplayName, EndonymOfficialName, CountryIso31661Alpha2Code, Country, ParentSubdivisionIso31662Code, ParentSubdivision (multiple levels of subdivisions), Type, Deprecation
Language:
Iso6391Code, DisplayName, Endonym, WritingDirection, IsSupported (I use this flag because I don't want to maintain translation catalogs for literally every language), SpokenInCountryIso31661Alpha2Codes, SpokenInCountries, LocaleIetfBcp47Tags, Locales (["en-US", "en-CA"]), Deprecation
Currency:
Iso4217AlphaCode, Iso4217NumericCode, DisplayName, OfficialName, DecimalPlaces, Symbol, IsSupported, AcceptedInCountryIso31661Alpha2Codes, AcceptedInCountries, Deprecation
Timezone:
IanaName, DisplayName, LocalizedDisplayNames, CurrentStdOffsetMinutes, CurrentDstOffsetMinutes, CurrentStdAbbrev, CurrentDstAbbrev, PrimaryCountryIso31661Alpha2Code, PrimaryCountry, CoApplicableCountryIso31661Alpha2Codes, CoApplicableCountries, Selectable, Aliases, Deprecation
...I'll stop there.
#2 Resilience
The most useful for those of you who like to share your hard-earned money with Azure, AWS or GCP (if you choose that one for some reason). Or, maybe you just self-host because you're a chad. Either way, this is the resilience mechanism for D2. It covers retries, circuit-breakers, singleflight, timeouts and concurrency rate-limiting. It also comes with a pipeline out-of-the-box (hahaha) that allows you to cleanly chain them together, override any defaults, change the order of operations, etc.
Here are some preferred configurations by use case:
Registration
Idempotent read-by-key over wire
// Use case: D2.Files → Edge context resolution, JWKS fetch, reference-data lookups.
// Key is the entity ID / IP / etc. — many concurrent callers for the same key are common.
services.AddKeyedSingleton<Singleflight<string, T>>(key);
services.AddKeyedSingleton<CircuitBreaker<T>>(key, (_, _) => new(_ => false));
services.AddResilientPipeline<string, T>(key, p => p
.UseSingleflight(key)
.UseRateLimiter(new RateLimiterOptions(maxConcurrency: 20))
.UseTimeout(new TimeoutOptions(TimeSpan.FromSeconds(30))) // total budget
.UseRetries(new()
{
MaxAttempts = 5,
BaseDelayMs = 500,
MaxDelayMs = 10_000,
})
.UseCircuitBreaker(key)
.UseTimeout(new TimeoutOptions(TimeSpan.FromSeconds(5)))); // per-attempt
// Retry outside CB (restart-recovery): MaxAttempts × backoff MUST exceed CooldownDuration.
// With 5 attempts and 500ms base/×2: ≈ 0.5 + 1 + 2 + 4 = 7.5s avg → fits 30s default cooldown.Standard service-to-service call
// Use case: any critical inter-service gRPC call in D2 (not idempotent-by-key,
// or SF not needed). Mirrors the strategy set of .NET's standard HTTP resilience handler (retry + circuit-breaker + rate-limiter + per-attempt timeout).
services.AddKeyedSingleton<CircuitBreaker<T>>(key, (_, _) => new(_ => false));
services.AddResilientPipeline<string, T>(key, p => p
.UseRateLimiter(new RateLimiterOptions(maxConcurrency: 20))
.UseTimeout(new TimeoutOptions(TimeSpan.FromSeconds(30)))
.UseRetries(new()
{
MaxAttempts = 3,
BaseDelayMs = 500,
MaxDelayMs = 10_000,
})
.UseCircuitBreaker(key)
.UseTimeout(new TimeoutOptions(TimeSpan.FromSeconds(8))));
// Restart-recovery without SF: each caller runs its own retry sequence.
// Add UseSingleflight(key) outermost if the call is a read-by-key hot path.At the call site
public sealed partial class FindWhoIs(
[FromKeyedServices(IpinfoServiceKeys.LOOKUP)] ResilientPipeline<string, IpinfoLookupResponse> pipeline,
IIpinfoClient ipinfo) : BaseHandler<...>
{
public override async ValueTask<D2Result<WhoIsDTO?>> ExecuteAsync(I input, CancellationToken ct)
{
var responseR = await pipeline.ExecuteAsync(
$"whois:{input.IpAddress}",
c => ipinfo.LookupAsync(input.IpAddress, c),
ct);
if (responseR.BubbleOnFailure<IpinfoLookupResponse, WhoIsDTO?>(out var bubbled, out var resp))
return bubbled;
return D2Result<WhoIsDTO?>.Ok(resp.ToDto());
}
}Why not _ ?
Microsoft.Extensions.Http.Resilience is HTTP only and Polly has no singleflight. A good portion of my outbound calls are not HTTP. They're RabbitMQ, EF Core, Redis, Internal Handlers, Blob Storage, gRPC, et cetera. Also, I wanted to preserve as much of the programming interface between my C# and TS side as possible. If you wanna learn more, here's the ADR for resilience in D2.
#N More?
Beyond what I already showed off I have a lot of other libraries that ship for both TypeScript and C# so you can pass around values and encrypted messages between Node and .NET:
- Results (errors as RICH values): NuGet, npm
- Encryption (symmetric and asymmetric; AES-256 GCM, P-256 ECDH-ES, HKDF-SHA256): NuGet, npm
- Tiered Caching (local in-memory and distributed; get, get many, exists, set, set many, increment, acquire lock, release lock, set and broadcast, backplane subscriptions): NuGet, npm
Comments (0)
No comments yet. Be the first to comment!