Technical glitches in Stock Brokers Electronic Trading Systems

Framework to address the ‘technical glitches’ in Stock Brokers’ Electronic Trading Systems

1. Rapid technological developments have increased the ease of electronic trading in securities markets. Technology related interruptions and glitches (technical glitches) and their impact on the investors’ opportunity to trade constitutes major technology related risk. Considering the growing number of such incidents, SEBI constituted a working group to recommend suitable measures to address the issue. Based on the recommendations of working group and views obtained from stakeholders & industry experts, it has been decided to put in place the following framework to deal with technical glitches occurring in the trading systems of stock brokers.
   
2. Definition of Technical Glitch:
   
   2.1 Technical glitch shall mean any malfunction in the systems of stock broker including malfunction in its hardware, software, networks, processes or any products or services provided by the stock broker in the electronic form. The malfunction can be on account of inadequate Infrastructure / systems, cyber-attacks / incidents, procedural errors and omissions, or process failures or otherwise, in their own systems or the one outsourced from any third parties, which may lead to either stoppage, slowing down or variance in the normal functions / operations / services of systems of the stock broker for a contiguous period of five minutes or more.
   
3. Reporting Requirements:
   
   3.1 Stock brokers shall inform about the technical glitch to the stock exchanges immediately but not later than 1 hour from the time of occurrence of the glitch.
   3.2 Stock brokers shall submit a Preliminary Incident Report to the Exchange within T+1 day of the incident (T being the date of the incident). The report shall include the date and time of the incident, the details of the incident, effect of the incident and the immediate action taken to rectify the problem.
   3.3 Stock brokers shall submit a Root Cause Analysis (RCA)Report (as per Annexure I) of the technical glitch to stock exchange, within 14 days from the date of the incident.
   3.4 RCA report submitted by the stock brokers shall, inter-alia, include time of incident, cause of the technical glitch (including root cause from vendor(s), if applicable), duration, chronology of events, impact analysis and details of corrective/ preventive measures taken (or to be taken), restoration of operations etc.
   3.5 Stock brokers shall submit information stated in para 3.1,3.2 and 3.3 above, by e-mail at [email protected], a common email address for reporting across all stock exchanges.
   3.6 All technical glitches reported by stock brokers as well as independently monitored by stock exchanges, shall be examined collectively by the stock exchanges alongwith the report/RCAand appropriate action shall be taken.
   
4. Capacity Planning:
   
   4.1 Increasing number of investors may create additional burden on the trading system of the stock broker and hence, adequate capacity planning is prerequisite for stock brokers to provide continuity of services to their clients. Stock brokers shall do capacity planning for entire trading infrastructure i.e server capacities, network availability,and the serving capacity of trading applications.
   4.2 Stock brokers shall monitor peak load in their trading applications, servers and network architecture. The Peak load shall be determined on the basis of highest peak load observed by the stock broker during a calendar quarter. The installed capacity shall be at least 1.5 times (1.5x) of the observed peak load.
   4.3 Stock brokers shall deploy adequate monitoring mechanisms within their networks and systems to get timely alerts on current utilization of capacity going beyond permissible limit of 70% of its installed capacity.
   4.4 To ensure the continuity of services at the primary data center, stock brokers as may be specified from time to time by stock exchange (hereafter referred to as specified stock brokers)shall strive to achieve full redundancy in their IT systems that are related to trading applications and trading related services.
   4.5 Stock exchanges shall issue detailed guidelines with regard to frequency of capacity planning to review available capacity, peak load, and new capacity required to tackle future load on the system.
   
5. Software testing and change management
   
   5.1 Software applications are prone to updates/changes and hence, it is imperative for the stock brokers to ensure that all software changes that are taking place in their applications are rigorously tested before they are used in production systems. Software changes could impact the functioning of the software if adequate testing is not carried out. In view of this, stock brokers shall adopt the following framework for carrying out software related changes / testing in their systems:
   5.2 Stock brokers shall create test driven environments for all types of software developed by them or their vendors. Regression testing, security testing and unit testing shall be included in the software development, deployment and operations practices.
   5.3 Specified stock brokers shall do their software testing in automated environments.
   5.4 Stock Brokers shall prepare a traceability matrix between functionalities and unit tests, while developing any software that is used in trading activities.
   5.5 Stock brokers shall implement a change management process to avoid any risk arising due to unplanned and unauthorized changes for all its information security assets (hardware, software, network, etc.).
   5.6 Stock brokers shall periodically update all their assets including Servers, OS, databases, middleware, network devices, firewalls, IDS /IPS desktops etc. with latest applicable versions and patches.
   5.7 Stock exchanges shall issue detailed guidelines with regard to testing of software, testing in automated environments, traceability matrix, change management process and periodic updation of assets etc.
   
6. Monitoring mechanism:
   
   6.1 Proactively and independently monitoring technical glitches shall be one of the approaches in mitigating the impact of such glitches. In this context, the stock exchange shall build API based Logging and Monitoring Mechanism (LAMA) to be operated between stock exchanges and specified stock brokers’ trading systems. Under this mechanism, specified stock brokers shall monitor key systems & functional parameters to ensure that their trading systems function in a smooth manner. Stock exchanges shall, through the API gateway, independently monitor these key parameters to gauge the health of the trading systems of the specified stock brokers.
   6.2 Stock Exchanges shall identify the key parameters in consultation with stock brokers. These key parameters shall be monitored by specified stock brokers and by stock exchanges, on a real time or on a near real time basis.
   6.3 Stock exchanges shall maintain a dedicated cell for monitoring the key parameters and the technical glitches occurring in stock brokers’ trading systems. The cell also shall intimate the specified stock broker concerned immediately about the breach of the key parameters monitored under LAMA.
   6.4 Stock brokers and stock exchanges shall preserve the logs of the key parameters for a period of 30 days in normal course. However, if a technical glitch takes place, the data related to the glitch, shall be maintained for a period of 2 years.
   
7. Business Continuity Planning (BCP) and Disaster Recovery Site (DRS):
   
   7.1 Stock brokers with a minimum client base across the exchanges, as may be specified by stock exchanges from time to time, shall mandatorily establish business continuity/DR set up.
   7.2 Stock brokers shall put in place a comprehensive BCP-DR policy document outlining standard operating procedures to be followed in the event of any disaster. A suitable framework shall be put in place to constantly monitor health and performance of critical systems in the normal course of business. The BCP-DR policy document shall be periodically reviewed to minimize incidents affecting the business continuity.
   7.3 The DRS shall preferably be set up in different seismic zones. In case, due to any reasons like operational constraints, such a geographic separation is not possible, then the Primary Data Centre (PDC) and DRS shall be separated from each other by a distance of at least 250 kilometers to ensure that both of them do not get affected by the same natural disaster. The DR site shall be made accessible from primary data center to ensure syncing of data across two sites.
   7.4 Specified stock brokers shall conduct DR drills / live trading from DR site. DR drills / live trading shall include running all operations from DRS for at least 1 full trading day. Stock exchanges in consultation with specified stock brokers shall decide the frequency of DR drill / live trading from DR site.
   7.5 Stock brokers, shall constitute responsible teams for taking decisions about shifting of operations from primary site to DR site, putting adequate resources at DR site, and setting up mechanism to make DR site operational from primary data center etc.
   7.6 Hardware, system software, application environment, network and security devices and associated application environments of DRS and PDC shall have one-to-one correspondence between them. Adequate resources shall be made available at all times to handle operations at PDC or DRS.
   7.7 Stock exchanges in consultation with stock brokers shall decide upon Recovery Time Objective (RTO) i.e.the maximum time taken to restore operations from DRS after declaration of Disaster and, Recovery Point Objective (RPO) i.e.the maximum tolerable period for which data might be lost due to a major incident.
   7.8 Replication architecture, bandwidth and load consideration between the DRS and PDCshallbe within stipulated RTO and the whole system shall ensure high availability, right sizing, and no single point of failure. Any updates made at the PDC shall be reflected at DRS immediately.
   7.9 Specified stock brokers shall obtain ISO certification as may be specified by stock exchanges from time to time in the area of IT and IT enabled infrastructure/processes of the stock brokers.
   7.10 The System Auditor, while covering the BCP –DR as a part of mandated annual System Audit, shall check the preparedness of the stock broker to shift its operations from PDC to DRS and also comment on documented results and observations on DR drills conducted by the stock brokers.
   7.11 Stock exchanges shall define the term ‘critical systems’, ‘disaster’ and issue detailed guidelines with regard to review of BCP document, DR drill/live trading, operating DR site from PDC, timeline for obtaining ISO certification etc.
   
8. Stock exchanges shall put in place a structure of financial disincentives applicable to stock brokers for technical glitches occurring in their trading systems and non-compliance of the provisions made in this regard.
   
9. Stock exchanges shall disseminate on their websites the instances of Technical glitches occurred in the trading system of stock brokers along with Root Cause Analysis (RCA) on such glitches.
   
10. Stock exchanges shall build necessary systems for implementation of the provisions of this circular and issue appropriate guidelines to the stock brokers for compliance with the provisions of this circular.
    
11. This circular is being issued in exercise of powers conferred under Section 11 (1) of the Securities and Exchange Board of India Act, 1992 to protect the interests of investors in securities and to promote the development of, and to regulate the securities market.
    
12. This circular shall come into effect from April 01, 2023.

Annexure

Read More on SEBI