nq
nq sdk usage guide for use with Testing Framework
The nq command is used to start the nq process either in interactive mode or to execute a script.
nq stands for 'Node.js query', which reflects its original use as a tool for quickly querying shared memory. (Alternatively, for users familiar with the Market Database, it might stand for 'not q'.)
Interactive REPL
If run with no arguments, an interactive REPL is launched:
$ ./nq
2019-11-21T04:55:43.088Z [info] Connecting to transaction socket: ipc:///home/yudhi/work/trunk-2.0/install/var/sockets/yudhi_11000_transaction_raw_socket_11200
2019-11-21T04:55:43.090Z [info] Connecting to blob transaction socket: ipc:///home/yudhi/work/trunk-2.0/install/var/sockets/yudhi_11000_transaction_blob_socket_11600
2019-11-21T04:55:43.090Z [info] Connecting to shared memory instance: yudhi_11000
2019-11-21T04:55:43.091Z [info] Connecting to MDB process 'gateway' at localhost:13005
2019-11-21T04:55:43.092Z [info] Attached to shared memory
2019-11-21T04:55:43.092Z [info] Reading system private key at /home/yudhi/work/trunk-2.0/install/etc/mg_systemkey
2019-11-21T04:55:43.094Z [info] Connected to MDB process: gateway
2019-11-21T04:55:43.123Z [info] Connected to transaction server
yudhi_11000>
This provides a quick environment for:
reading from shared memory; or
entering a sandbox for a particular user in order to see what that user can see, or to execute a transaction as that user.
Some tab-completion is provided. Pressing the TAB key twice will show all items in scope:
yudhi_11000>
Account AccountType ActualBuySellValueAdjustStatus ActualBuySellValueAdjustType Aggressor
Alert AlertCategory AlertChannel AlertConfiguration AlertConfiguration_Status
AlertDefinition AlertDefinitionDerivedColumn AlertDefinitionDestination AlertDefinition_FrequencyPeriod AlertDestination
...
Typing the beginning of an identifier then pressing TAB will complete the word; TAB again will display possible completions.
Typing .help will list meta-commands which can be executed.
Typing the name of a table will show the contents of that table:
yudhi_11000> Firm
Id ShortName Status FirmType TemplateInstanceFirmType FeeSet CheckAccounts TrackHoldings CanShortSell CanResellToday TrackCash CanUseTodaysCash CashCanBeNegative Enterprise Name CheckLimits WarnPercent LimitStatus..
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
-1 SYSTEM 0x0 0 0 null false false false false false false false SYSTEM SYSTEM false Ok ..
1 MarketGrid 0x3ff System 0 null false false false false false false false SYSTEM MarketGrid false Ok ..
2 @Broker 0x10003ff Template Regular null false true false false false false false SYSTEM Broker false Ok ..
20001 CVEX 0x3ff Exchange 0 null false false false false false false false CVEX CVEX false Ok ..
30101 FirmA 0x3ff Regular 0 null true true false true false true false A Firm A true Ok ..
30201 FirmB 0x3ff Regular 0 null true true false true false true false B Firm B true Ok ..
30301 FirmC 0x3ff Regular 0 null true true false true false true false C Firm C true Ok ..
30401 FirmD 0x3ff Regular 0 null true true false true false true false D Firm D true Ok ..
30501 FirmE 0x3ff Regular 0 null true true false true false true false E Firm E true Ok ..
30601 FirmF 0x3ff Regular 0 null true true false true false true false F Firm F true Ok ..
30701 FirmG 0x3ff Regular 0 null true true false true false true false G Firm G true Ok ..
30801 FirmH 0x3ff Regular 0 null true true false true false true false H Firm H true Ok ..
Entering a sandbox in the REPL
For debugging, it is often convenient to enter a sandbox for a particular user.
Type .su followed by the UserId of the user to 'become' that user. The prompt will change to indicate the current sandbox:
yudhi_11000> .su userA
userA@yudhi_11000>
Now typing the name of a table will show the sandboxed version of that table:
userA@yudhi_11000> Firm
Id ShortName Status FirmType TemplateInstanceFirmType FeeSet CheckAccounts TrackHoldings CanShortSell CanResellToday TrackCash CanUseTodaysCash CashCanBeNegative Enterprise Name CheckLimits WarnPercent LimitStatus Electro..
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
30101 FirmA 0x3ff Regular 0 null true true false true false true false A Firm A true Ok null ..
This is useful for determining what records a particular user can view.
Sending transactions
Once within a sandbox, it is possible to send transactions to the engine as the current user.
First, type .login. This causes a LoginJwt transaction to be sent. The JWT is automatically generated using the system private key; no password is required, since this can only be done with shell access to the machine itself.
Prior to version 2.6, a password was required:
.login <password>
After logging in, transactions may be sent using the .tx command.
To list all possible transactions, use .tx with no arguments (since version 2.6):
userA@master> .tx
OrderNew (1001) OrderHitLift (1046) RfqRespond (1078) ApplicationTimeoutSet (1106) SetSystemId (1197)
OrderAmend (1002) NegDealConfirm (1047) RfqAccept (1079) StaticSet (1107) SnapshotRate (1198)
OrderStatusSet (1003) TwoSideReport (1048) RfqReject (1080) AddEnterprise (1108) CycleSystem (1199)
To view the parameters of a transaction, type .tx followed by the transaction name. This will list the optional and required fields:
userA@master> .tx AddFirm
AddFirm { field: val, ... }
Required fields:
enterprise SYMBOL (=> Enterprise)
shortname char
name char
industry SYMBOL (=> Industry)
status Status
feeset SYMBOL (=> FeeSet)
checkaccounts CODE_S
...
Optional fields:
templatefirm SYMBOL (=> Firm)
settlementcurrency SYMBOL
warnpercent VALUE
buyvollimit VALUE
sellvollimit VALUE
...
Finally, to enter the transaction, type .tx followed by the fields and values in JavaScript object literal format ({key: value, …}).
Switching users and exiting nq
To switch to another user, simply type .su followed by the desired user ID, or no user ID to leave the sandbox altogether. If logged in, nq will automatically log out after a short period of time, or when the process is exited. (Type .logout to log out immediately.)
To exit the REPL, press CTRL+D.
Running scripts
In addition to the REPL environment, nq may be used to execute ECMAScript files. In this configuration it becomes a highly flexible tool for executing either one-off tasks or long-running processes.
Script files are located within the directory lib/ui/server/scripts/ inside the MarketGrid release directory.
The set of available scripts may be listed using the command nq --list (since version 2.6):
$ ./nq --list
entity_status
funding
generate_jwt
mark_price_bot
order_bot
To run a script, use nq --run <script-name>, where script-name is one of the listed scripts.
A script may declare its own set of command-line options, in which case they may be listed using the --help argument (together with --run):
$ ./nq --run mark_price_bot --help
Update the MarkPrice of an InstrumentMarket on a given interval.
Available options:
--user User ID used to enter the mark price (default: admin)
admin
--ticker InstrumentMarket to update (default: BTCUSD-Perp)
BTCUSD-Perp
--freq Update frequency in milliseconds (default: 1000)
1000
Examples
Example : View all Orders placed
test-user@test-run-29> Order
Id Status Timestamp Reference InstrumentMarket InstrumentType Market OriginalOrder AmendedOrder PairedOrd..
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 0x3 2021-01-28 22:50:45.249013500Z 1 test-market:test-im Standard test-market 1 ..
2 0x3 2021-01-28 23:02:33.440135800Z 2 test-market:test-im Standard test-market 2 ..
3 0x3 2021-01-28 23:01:58.245309000Z 3 test-market:test-im Standard test-market 3 ..
4 0x3 2021-01-28 23:02:33.440135800Z 4 test-market:test-im Standard test-market 4 ..
Similarly Trades command will show all Trades executed
Example : View all Orders placed only specified fields
test-user@test-run-29> .print Order only Id Reference InstrumentMarket Price Currency TotalQuantity TotalBalance
Id Reference InstrumentMarket Price Currency TotalQuantity TotalBalance
────────────────────────────────────────────────────────────────────────────────
1 1 test-market:test-im 100 test-currency 1 1
2 2 test-market:test-im 102 test-currency 1
3 3 test-market:test-im 103 test-currency 1 1
4 4 test-market:test-im 102 test-currency 1
Example : View Orders with specific condition
Orders with outstanding quantity
test-run-29> .print Order.where(gt("TotalBalance",0)) only Id InstrumentMarket Side Price TotalQuantity TotalBalance
Id InstrumentMarket Side Price TotalQuantity TotalBalance
─────────────────────────────────────────────────────────────
1 test-market:test-im Buy 100 1 1
3 test-market:test-im Sell 103 1 1
Fully traded orders
test-run-29> .print Order.where(eq("TotalBalance",0)) only Id InstrumentMarket Side Price TotalQuantity TotalBalance
Id InstrumentMarket Side Price TotalQuantity TotalBalance
─────────────────────────────────────────────────────────────
2 test-market:test-im Sell 102 1
4 test-market:test-im Buy 102 1
Partially traded orders
test-run-29> .print Order.where(any(...Order.rows.filter(o => o.TotalQuantity > o.TotalBalance && o.TotalBalance > 0).map(o => eq('Id', o.Id)))) only Id Instrument InstrumentType Price TotalQuantity TotalBalance
Id Instrument InstrumentType Price TotalQuantity TotalBalance
───────────────────────────────────────────────────────────────────
5 test-instrument Standard 104 10 9
Example using like condition
test-run-28> User
Id Status UserId UserType TemplateUser LoggedIn MaxLogins FailedLogins MaxFailedLogins LockedOut PasswordMustChange Firm Groups WorkingStatus PasswordExpiryDa..
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
-1 0x0 System 255 false false System 1 0x0 0000-00-00 ..
1 0x3 admin Exchange false false test-firm 0x3 0000-00-00 ..
2 0x3f test-user Regular false false test-firm 0x3f 0000-00-00 ..
3 0x3f demo-user Regular false false test-firm 0x3f 0000-00-00 ..
test-run-28> User.where(like('UserId',"dem"))
Id Status UserId UserType TemplateUser LoggedIn MaxLogins FailedLogins MaxFailedLogins LockedOut PasswordMustChange Firm Groups WorkingStatus PasswordExpiryDa..
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
3 0x3f demo-user Regular false false test-firm 0x3f 0000-00-00 ..
test-run-28>
Example with sort
Sorting the User table as per UserId
test-run-28> .print User only Id UserId UserType
Id UserId UserType
──────────────────────
-1 System
1 admin Exchange
2 test-user Regular
3 demo-user Regular
test-run-28> .print User.sort(asc('UserId')) only Id UserId UserType
Id UserId UserType
──────────────────────
-1 System
1 admin Exchange
3 demo-user Regular
2 test-user Regular
test-run-28>
Example : Maintaining StaticData
Static Data Tables can be managed by StaticUpdate transactions
You will need permissions to manage
test-run-6730> .su admin
admin@test-run-6730>
Creating a new User
.tx StaticUpdate User { UserId: 'demo-user-2', UserType: 100, Firm: 1, ShortName: 'demo-user-2', Name: 'demo-user-2'}
Update a User with Id
.tx StaticUpdate User {Id:3,Name:'demo-usr-4'}
Update a User based on other fields by finding Id
.tx StaticUpdate User {Id:User.find(eq('UserId','demo-usr-4')).Id,Name:'ChangingName'}