type-mapping
Map input data safely
type-mapping
Never trust incoming data. Always check it.
This package was written to help check/sanitize/map data declaratively and comes with batteries included.
Each Mapper
is a function.
Goals
-
Usability
It must be easy for developers to understand and use.
Complex
Mapper
s are created using function composition by default. However, a fluent API also exists and may be used instead. -
Extensibility
Easy for developers to create their own custom
Mapper
s, and use them with the default ones.All
Mapper
s are just functions. Creating aMapper
is as easy as creating a function has certain properties. -
Composability
Developers should be able to integrate this package into their own packages to boost run-time type safety.
-
Use Case Coverage
The default
Mapper
s should cover 95+% of common data mapping scenarios.They should also facilitate building custom
Mapper
s, if they cannot satisfy a custom use case.Common use cases,
- Mapping data from/to incoming HTTP requests/API calls/websocket events
- Mapping data from/to "schema-on-write" databases (e.g. MySQL)
- Mapping data from/to "schema-on-read" databases (e.g. Mongo)
-
node
and Browser supportThis package should work on as many
node
and Browser environments as possible. This package should have as few run-time dependencies as possible (zero at the moment). -
Safety
This package must not map data that is invalid. Bugs are inevitable but they should be fixed and tested for as quickly as possible.
Non-Goals
-
Performance
Performance is somewhat a consideration. However, safety, correctness and usability have priority over performance.
-
Reflection
Installation
npm install --save type-mapping
Basic Usage
; ;/* OK! [1,2,3,4,5]*/arr"a-name-identifying-this-array", ;/* OK! [1,2,3,4,5]*/arr"string-to-unsigned-int", ;/* Error thrown: invalid[0] is not an unsigned integer, and cannot be cast*/arr"invalid", ; ;/* OK!*/user"john", ;/* Error thrown: invalid.userId must be bigint; received string*/user"invalid", ; ;//OKstringOrUndefined"is-string", "hello";//OKstringOrUndefined"is-undefined", undefined;//ErrorstringOrUndefined"is-54", 54;
See the test
directory for more examples.
Fluent API
The stringOrUndefined
example is simple and shows that
you may combine Mapper
s.
However, with more complex Mapper
s, function composition
gets unwieldly.
A fluent API exists that may be used.
//We changed the import to "type-mapping/fluent"; ;//OKstringOrUndefined"is-string", "hello";//OKstringOrUndefined"is-undefined", undefined;//ErrorstringOrUndefined"is-54", 54;
A more complex example,
;/* Expected Input: ( { anEnum: E; requiredStringButMayBeUndefined: string | undefined; stringOrNumber: string | number; nullableMysqlDateTime: Date | null; nullableMysqlDateTime2: Date | null; arrayOfEnumKey: ("A" | "B" | "C")[]; literal_1_or_5: 1 | 5; nullOrNonEmptyString: string | null; } & { optionalString?: string | undefined; } )[] Output: { anEnum: E; optionalString: string | undefined; requiredStringButMayBeUndefined: string | undefined; stringOrNumber: string | number; nullableMysqlDateTime: Date | null; nullableMysqlDateTime2: Date | null; arrayOfEnumKey: ("A" | "B" | "C")[]; literal_1_or_5: 1 | 5; nullOrNonEmptyString: string | null; }[]*/;
Recommended Usage
As much as possible, use "type-mapping/fluent"
and its fluent API.
You should only really need to use type-mapping
and its functional API
when building custom mappers, or writing a library that utilizes this package
for compile-time and run-time type safety.
As much as possible, avoid calling Mapper
s directly.
Use map()/mapMappable()
or tryMap()/tryMapMappable()
instead.
map()
is an alias for mapExpected()
.
tryMap()
is an alias for tryMapExpected()
.
//Fluent API;//HandledInput : unknown//MappableInput : string|number//ExpectedInput : number//Output : number; //== Calling `Mapper` directly == //Compile-time: OK//Run-time : OK; 1strToNum"x", "1";//Compile-time: OK//Run-time : OK; 1strToNum"x", 1;//Compile-time: OK//Run-time : Error; x must be finite number, or finite number stringstrToNum"x", "hello";//Compile-time: OK//Run-time : Error; x must be finite number, or finite number stringstrToNum"x", true; //== Calling `map()/mapExpected()` == //Compile-time: Error; string not assignable to number//Run-time : OK; 1strToNum.map"x", "1";//Compile-time: OK//Run-time : OK; 1strToNum.map"x", 1;//Compile-time: Error; string not assignable to number//Run-time : Error; x must be finite number, or finite number stringstrToNum.map"x", "hello";//Compile-time: Error; boolean not assignable to number//Run-time : Error; x must be finite number, or finite number stringstrToNum.map"x", true; //== Calling `tryMap()/tryMapExpected()` == //Compile-time: Error; string not assignable to number//Run-time : { success : true, value : 1 }strToNum.tryMap"x", "1";//Compile-time: OK//Run-time : { success : true, value : 1 }strToNum.tryMap"x", 1;//Compile-time: Error; string not assignable to number//Run-time : Error; { success : false, err : Error("x must be finite number, or finite number string") }strToNum.tryMap"x", "hello";//Compile-time: Error; boolean not assignable to number//Run-time : Error; { success : false, err : Error("x must be finite number, or finite number string") }strToNum.tryMap"x", true; //== Calling `mapMappable()` == //Compile-time: OK//Run-time : OK; 1strToNum.mapMappable"x", "1";//Compile-time: OK//Run-time : OK; 1strToNum.mapMappable"x", 1;//Compile-time: OK//Run-time : Error; x must be finite number, or finite number stringstrToNum.mapMappable"x", "hello";//Compile-time: Error; boolean not assignable to string|number//Run-time : Error; x must be finite number, or finite number stringstrToNum.mapMappable"x", true; //== Calling `tryMapMappable()` == //Compile-time: OK//Run-time : { success : true, value : 1 }strToNum.tryMapMappable"x", "1";//Compile-time: OK//Run-time : { success : true, value : 1 }strToNum.tryMapMappable"x", 1;//Compile-time: OK//Run-time : Error; { success : false, err : Error("x must be finite number, or finite number string") }strToNum.tryMapMappable"x", "hello";//Compile-time: Error; boolean not assignable to string|number//Run-time : Error; { success : false, err : Error("x must be finite number, or finite number string") }strToNum.tryMapMappable"x", true;
With the functional API,
//Functional API;//HandledInput : unknown//MappableInput : string|number//ExpectedInput : number//Output : number; //Calling `Mapper` directly//Compile-time: OK//Run-time : Error; x must be finite number, or finite number stringstrToNum"c", true; //Calling `map()`//Compile-time: Error; boolean not assignable to number//Run-time : Error; x must be finite number, or finite number stringtm.mapstrToNum, "c", true; //Calling `tryMap()`//Compile-time: Error; boolean not assignable to number//Run-time : Error; { success : false, err : Error("x must be finite number, or finite number string") }tm.tryMapstrToNum, "c", true; //Calling `mapMappable()`//Compile-time: Error; boolean not assignable to string|number//Run-time : Error; x must be finite number, or finite number stringtm.mapMappablestrToNum, "c", true; //Calling `tryMapMappable()`//Compile-time: Error; boolean not assignable to string|number//Run-time : Error; { success : false, err : Error("x must be finite number, or finite number string") }tm.tryMapMappablestrToNum, "c", true;
When building a package that uses this package for type-safety,
try to avoid forcing users to invoke Mapper
s explicitly.
Instead, have them pass the Mapper
s to your package,
and have your package invoke them behind the scenes.
MappingError
A MappingError
is an Error
that implements a certain interface.
TODO, more documentation
A MappingError
should give you detailed information about why a mapping failed,
and should provide enough metadata to let you write a custom error handler.
When created with ErrorUtil.makeMappingError()
, all properties of MappingError
that are not properties of Error
are non-enumerable.
This means that they will not show up if you use Object.keys()
or JSON.stringify()
.
This is intentional because serializing all data about an error may be undesirable, especially if the error contains sensitive information.
You should use ErrorUtil.isMappingError()
to detect a MappingError
and
handle such errors explicitly.
Mapper
s
Default TODO, document all default mappers
The default mappers may be found in src/functional-lib
Mapper
s
MySQL TODO, document all default MySQL mappers
The default MySQL mappers may be found in src/mysql-lib
;;
Fields
TODO, talk about fields and Name<string>
Decorators
The following decorators are provided,
@prop(mapper : Mapper)
@setter(mapper : Mapper)
@method(...mappers : Mapper[])
func(...mappers : Mapper[])
func
is not quite a decorator but may be used to wrap a function.
@prop
The @prop(mapper : Mapper)
decorator is a generic property decorator
and takes a single Mapper
. This decorator may be used on class properties.
During run-time, the decorator creates a getter
and setter
on
each class instance. Attempts to set the value of the property
will pass the value through the Mapper
before setting the value.
;c.x = 1; //OKc.x = -1; //Error, expected unsigned integer or undefinedc.x = undefined; //OKc.x = null; //Error, expected unsigned integer or undefined
The @prop
decorator also works with class inheritance,
; c.prop0; //9c.prop0 = 5; //OKc.prop0 = 4; //Error, expected > 4c.prop0 = 3; //Error, expected > 4c.prop0 = 2; //Error, expected > 4c.prop0 = 1; //Error, expected > 4
@setter
The @setter(mapper : Mapper)
decorator is a generic accessor decorator
and takes a single Mapper
. This decorator may be used on class setters.
During run-time, the decorator replaces the setter
on the class prototype. Attempts to set the value of the property
will pass the value through the Mapper
before being passed
to the setter
.
;;c.x = 1; //OKc.x = -1; //Error, expected unsigned integer or undefinedc.x = undefined; //OKc.x = null; //Error, expected unsigned integer or undefined
@method
The @method(...mappers : Mapper[])
decorator is a generic method decorator
and takes a zero to many Mapper
s. This decorator may be used on class methods.
During run-time, the decorator replaces the method's value
on the class prototype. Attempts to call the method
will pass the value through the Mapper
s before being passed
to the method.
;; c.foo5;value; //5c.foo"6" as any; //Error, expected finite numbervalue; //5
Rest parameters are also supported, with the following syntax,
;;; c.foo5; //OKvalue; //5arr; //[] c.foo6, "a", "b", "c"; //OKvalue; //6arr; //["a", "b", "c"]c.foo7, "a", "b", "c", 5 as any; //Error, expected string in argument 4
func
The func(...mappers : Mapper[])
function is not quite a decorator.
It is a generic function that takes a zero to many Mapper
s.
The result may be used to wrap other functions and map their arguments before calling the wrapped functions.
;; foo5; //OKvalue; //5foo"6" as any; //Error, expected finite number
Rest parameters are also supported, with the following syntax,
;;//Notice the `...[tm.string()]` syntax foo5; //OKvalue; //5arr; //[]foo"6" as any; //Error, expected finite number foo6, "a", "b", "c"; //OKvalue; //6arr; //["a", "b", "c"]foo7, "a", "b", "c", 5 as any; //Error, expected string in argument 4
Mapper
s
Custom A Mapper
is anything that implements the Mapper<>
interface,
It is a function with two parameters.
name
is the name of the value being mapped.
mixed
is usually a value of type unknown
.
Extra care is needed to correctly map (or reject) values of type unknown
.
Usually, you should implement SafeMapper<>
,
;
All Mapper<HandledInputT, OutputT>
types must satisfy the following properties,
-
Correctness
When given an input of type
HandledInputT
, it must handle it correctly.It must not silently produce invalid output.
It must not throw an
Error
on valid input.If you pass an input that is not of type
HandledInputT
, the behaviour is undefined. -
Idempotence
A
Mapper<>
must satisfy the following,deepEqualsfx, ffxThat is, if
f(0)
is"hello"
, thenf("hello")
must be"hello"
. -
Immutability
A
Mapper<>
must NEVER modify its input argument.That is, all inputs must be treated as immutable.
A
Mapper<>
may,- Return the same input
- Return a copy of the input
- Return a completely different object
An example of a custom mapper,
;
Now, you may use the mapper as-is, or compose it with other mappers,
;; evenNumberOnly"x", 3; //Error, x must be an even numberevenNumberOnly"x", 4; //OKevenNumberOnly"x", "qwerty"; //Error, x must be a number evenNumberOrString"x", 3; //Error, x must be an even numberevenNumberOrString"x", 4; //OKevenNumberOrString"x", "qwerty"; //OK
Buffer
support
TODO, how Buffer
is supported on environments that do not support Buffer
.
BigInt
support
This package supports environments that have bigint
primitive support.
This package tries its best to support environments that use a BigInt
polyfill.
If using a polyfill, the polyfill must satisfy the following,
-
Implement
function BigInt (value : string|number|bigint) : Object
on the global scope (window
/global
/globalThis
).It must return an
Object
. Not a primitive. -
Implement
.toString()
The simplest supported polyfill, with no error checking, is,
global as any.BigInt = as any;
Optional
You may make a mapper optional with optional<>()
,
;;;obj"obj", ; //OK; { x : undefined }obj"obj", ; //OK; { x : undefined }obj"obj", ; //OK; { x : "a" }obj"obj", ; //OK; { x : "b" }obj"obj", ; //Error; obj.x must be "a"|"b"
Using optional<>()
cancels out runTimeRequired<>()
orUndefined<>()
orUndefined<>()
may have surprising behaviour.
Using orUndefined<>()
on a field has the following effect,
- The field is required during compile-time
- The field is optional during run-time
This behaviour lets us use the mappers that map undefined
to play nicely with JSON serialization.
//Current behaviour//Compile-time required//Run-time optional;;; //{ x : undefined }; //"{}"; //{}; //{ x : undefined }
vs.
//Technically more "correct" behaviour but harder to use with JSON serialization//Compile-time required//Run-time required;;; //{ x : undefined }; //"{}"; //{}; //Error; output must have property "x"
To make a field allow undefined
but also make the field required,
;tm.runTimeRequiredtm.orUndefinedmyMapper
Or, with the fluent API,
;myMapper.orUndefined.runTimeRequired;
Then, you'll get the following result,
;;;obj"obj", ; //OK; { x : undefined }obj"obj", ; //OK; { x : undefined }obj"obj", ; //OK; { x : "a" }obj"obj", ; //OK; { x : "b" }obj"obj", ; //Error; obj.x must be "a"|"b" ;obj"obj2", ; //Error; obj must have property "x"obj"obj2", ; //OK; { x : undefined }obj"obj2", ; //OK; { x : "a" }obj"obj2", ; //OK; { x : "b" }obj"obj2", ; //Error; obj2.x must be "a"|"b"
RunTimeRequired
You may make a mapper required during run-time with runTimeRequired<>()
,
;;;obj"obj", ; //OK; { x : undefined }obj"obj", ; //OK; { x : undefined }obj"obj", ; //OK; { x : "a" }obj"obj", ; //OK; { x : "b" }obj"obj", ; //Error; obj.x must be "a"|"b" ;obj2"obj2", ; //Error; obj2 must have property "x"obj2"obj2", ; //OK; { x : undefined }obj2"obj2", ; //OK; { x : "a" }obj2"obj2", ; //OK; { x : "b" }obj2"obj2", ; //Error; obj2.x must be "a"|"b"
Using runTimeRequired<>()
cancels out optional<>()
but does not remove
undefined
from the HandledInput
/MappableInput
/ExpectedInput
.
HandledInput
vs MappableInput
vs ExpectedInput
In general,
ExpectedInput
extendsMappableInput
MappableInput
extendsHandledInput
The HandledInput
of a Mapper
is the range of values
it will handle correctly.
If given a type inside the range of HandledInput
,
it must throw an Error
on invalid input, and
must map valid inputs correctly.
If you pass a type outside the range of HandledInput
,
the behaviour is undefined.
It may behave as intended, or do something else completely.
The MappableInput
of a Mapper
is the range of values
it will (probably) map successfully.
TypeScript's type system may not be strong enough to fully express the range of values that will map successfully.
But, in an ideal world, passing a type inside the range
of MappableInput
will always guarantee a successful
mapping with zero Error
s thrown.
The ExpectedInput
of a Mapper
is the range of values
it would ideally like to receive.
For example, a Mapper
may map string
values to number
,
and perform no mapping on number
values.
The MappableInput
is string|number
.
However, in your application logic,
you may expect to never pass number
to the Mapper
.
So, you set the ExpectedInput
to string
.
; declare ; //OKtm.tryMapExpectedstringToNumber, "x", "23";//Compile-Error, expected string//Even though this will work during run-timetm.tryMapExpectedstringToNumber, "x", 23;//Compile-Error, expected string//Will also throw an `Error` during run-timetm.tryMapExpectedstringToNumber, "x", true; //OKtm.tryMapMappablestringToNumber, "x", "23";//OK, `number` is mappabletm.tryMapMappablestringToNumber, "x", 23;//Compile-Error, expected string|number//Will also throw an `Error` during run-timetm.tryMapMappablestringToNumber, "x", true; //OKtm.tryMapHandledstringToNumber, "x", "23";//OKtm.tryMapHandledstringToNumber, "x", 23;//OK, HandledInput is `unknown`.//So, it will handle `boolean` correctly...//By throwing an `Error` during run-time.tm.tryMapHandledstringToNumber, "x", true;
_debug
TODO, how to debug compile-time types
Mapper
operations
Mapper
predicates
Mapper
queries
EnumUtil
BigIntUtil
ErrorUtil
TypeUtil
Contributing
Tests
npm run sanity-check
The above command rebuilds this package and runs the compile-time and run-time tests.
schema-decorator
Compatibility with This package is the successor to schema-decorator
, and meant to replace it.
Many breaking changes have been made.
In particular, the concept of Accepts
and CanAccept
have been replaced with,
ExpectedInput
and MappableInput
respectively.
There is no Field
class anymore.
Instead, a Mapper
is a Field
when it also extends the Name<>
interface.
Apart from those breaking changes, you may use the mappers here with
the mappers from schema-decorator
.
The route declaration feature has also been removed and has been split into,
typed-orm
Other packages using -
https://github.com/anyhowstep/route-declaration
route-declaration
is used to declare HTTP routes -
https://github.com/anyhowstep/route-client
Uses
route-declaration
to provide compile-time and run-time type-safe API calls.Uses
axios
to send requests by default. May be extended to use other request senders. -
https://github.com/anyhowstep/route-express
Uses
route-declaration
to provide compile-time and run-time type-safe API servers.Is a thin wrapper over
express
.Provides compile-time type-safe
res.locals
manipulation.
-
https://github.com/anyhowstep/typed-orm
Experimental MySQL 5.7 query builder and ORM.
Wraps https://github.com/mysqljs/mysql and provides compile-time and run-time type-safe SQL expression composition, and query building.
Supports connection pooling and transactions.
-
https://github.com/AnyhowStep/tsql
Work-in-progress rewrite of
typed-orm
.Improved version of the experimental
typed-orm
.Intended to act as a database-agnostic base package to support multiple databases. (PostgreSQL, MariaDB, SQLite3, MySQL, etc.)
-
https://github.com/anyhowstep/tsql-mysql-5.7
Work-in-progress rewrite of
typed-orm
.Uses
tsql
to implement MySQL 5.7-specific operators, functions, and syntax.
Cookbook
The examples here use the fluent API.
(TODO, examples of common data mapping scenarios and how to handle them)
Breaking Changes
-
1.3.0 -> 1.4.0
optional()
still makes a field compile-time optional.optional()
still makes a field run-time optional.orUndefined()
now makes a field run-time optional. UserunTimeRequired()
to make it run-time required again. This breaking change was introduced to make usage with JSON serialization easier.runTimeRequired()
addedrunTimeRequired()
makes a field compile-time required.runTimeRequired()
makes a field run-time required.
-
1.20.0 -> 1.21.0
mysql.decimal()
now returns theDecimal
interface and notstring
. This breaking change was introduced because not much code should break and this increases type safety overall.
-
1.24.0 -> 1.25.0
mysql.xxxIntXxx()
functions now returnbigint
and notnumber
. This breaking change was introduced because this increases type safety overall. This also follows the general direction thetsql
project wants to follow.