Need for Independent Agency for Field Trials

Need for Independent Agency for Field Trials

Major General Mrinal Suman

Cancellation of the contract for the procurement of helicopters for VVIPs is the latest episode in the long saga of India’s aborted defence deals. The Indian defence procurement regime has acquired notoriety for its unprofessional, slipshod and flawed treatment of sensitive issues that affect nation’s operational preparedness.

Every single deal that is concluded in a competitive environment gets embroiled in allegations of skewed decision making and favouritism. In the case of VVIP helicopter deal, Comptroller and Auditor General of India (CAG) has, inter alia, faulted the Government for prejudiced formulation of the Services Qualitative Requirements (SQR), discriminatory trials and subjective staff evaluation. All the three activities were carried out under the aegis of the Air Headquarters. 

Selection of equipment for procurement is carried out in two stages. First, the equipment is subjected to technical evaluation to confirm that the equipment complies with all performance parameters that are considered essential for the equipment to be able to fulfill the envisaged mission. In the second stage, commercial quotes of all technically acceptable proposals are opened for inter se comparison, and the lowest bidder is identified for the award of the contract.  

  

Excessive Concentration of Powers

The Defence Procurement Procedure (DPP) has bestowed excessive decision making powers on a single agency, i.e. the concerned Service Headquarters (SHQ). SHQ prioritises a proposal, initiates it, formulates SQR, prepares initials list of vendors to be invited to submit proposals, carries out paper evaluation of proposals to short-list vendors for trials, prepares trial directive, selects trial units, oversees trials, compiles reports and prepares staff evaluation report. 

Validation of technical suitability of equipment on offer is by far the most critical process. It is a three stage progression – technical evaluation of proposals, field trials and staff evaluation. 

To start with, a broad based Technical Evaluation Committee (TEC) is constituted by the SHQ to carry out evaluation of technical proposals submitted by vendors to determine their compliance with the specified SQR and identify vendors who should be called for trials. 

Field trials are by far the most critical activity as performance claims made by the vendors in their proposals are validated in actual terrain and climatic conditions. Maintainability aspects are also examined. Due to the fact that a SHQ exercise total control over the process, field trials suffer from the following serious infirmities:-

·         Development of a trial plan requires scientific and methodical treatment for which a SHQ possesses no expertise. Consequently, trial directives are prepared in a routine manner by enumerating all parameters sequentially. Many aspects remain imprecise and are interpreted differently by various trial units. 

·         Trial teams are selected based on their location and not on the basis of the required competence. No effort is made to check whether they are appropriately equipped and trained to evaluate equipment with diverse technologies and determine their inter se merit. As a result, many trial units fail to comprehend the importance of trials and treat such duties as an added encumbrance.

·         Whereas no service equipment is ever used in isolation and its true potential can only be gauged when full-fledged exploitation trials are carried out in conjunction with associated systems, the current practice of nominating a trial unit to carry out validation of performance parameters serves limited purpose. It neglects the critical aspect of suitability of equipment for integration in the existing equipment profile for mission accomplishment. Similarly, inadequate attention is paid to the validation of the support system and maintainability aspects. 

·         Trial units lack expertise to prepare unambiguous, cogent and convincing trial reports.  Final recommendations are couched in generalities. Imperfect reports not only delay decision-making but even invite adverse comments as regards their objectivity. At times retrials get necessitated, resulting in cost and time overruns.

Finally, once necessary inputs are received from all the trial units and maintenance agencies, staff evaluation is carried out to collate, analyse and review data to determine whether a system satisfies its acceptance criteria. If convinced of the operational capability of the system, Staff Evaluation Report recommends its induction into service. Submission of the report to MoD marks culmination of the technical evaluation process.

 

Excessive concentration of decision making powers in the hands of a SHQ lends itself to easy manipulation by errant decision makers through the following means:-

·         As resources are always insufficient to cater for all procurement proposals, priorities can be skewed to accord higher precedence to a proposal in which a favoured vendor is confident of emerging successful.

·         SQR can be formulated in a vendor-specific manner to rule out competition and contriving a single vendor situation. 

·         Undue influence can be brought to bear on trial teams to take subjective view of competing equipment, especially in respect of trial parameters which are descriptive in nature and cannot be quantified.

·         In the staff evaluation, criticality of certain characteristics of the favoured equipment can be over-played and minor glitches of the rest can be highlighted to play favourites.

The Way Forward

No single agency or authority should be able to influence the complete process of technical evaluation. For that, decentralisation of different activities is essential to prevent fraudulent and corrupt practices. As initiation of proposal, evolution of SQR, short-listing of vendors for field trials and preparation of staff evaluation report have to be carried out at the SHQ, it will be prudent to assign the task of field trials to an independent authority. As stated earlier, the functionaries at SHQ neither possess needed expertise nor can spare adequate time to oversee field trials.

In most of the foreign militaries, trials are performed by specially constituted ‘trial groups’. Whereas France assigns the task to DGA, the US Army has raised Test and Evaluation Command for the purpose.

The US Army Test and Evaluation Command (ATEC) is responsible for developmental testing, independent operational testing, independent evaluations, assessments and experiments of army equipment to provide essential information to the acquisition decision makers. ATEC has three subordinate commands. Army Developmental Test Command conducts rigorous performance tests on weapon systems and material. Army Operational Test Command conducts realistic operational testing in the critical areas of equipment, doctrine, force design and training. Army Evaluation Centre writes the final report, used by the decision-makers to determine whether a new or enhanced system should be inducted into the US Army. 

The US Defence Acquisition University conducts courses in test and evaluation of military equipment at three levels - introductory, intermediate and advanced. As field trial of new equipment with diverse technologies is a highly specialised task, trial teams are appropriately equipped and trained. Additionally, they are given necessary expertise and vision to be able to interpolate exploitation of equipment in diverse combat milieu.

India should adopt the US model, albeit with modifications to suit its needs. For the Indian Army, the Training Command can effectively undertake the trial and evaluation responsibility as all major training establishments come under its functional control and they possess the best talent and infrastructural facilities. In addition, as the Training Command also deals with doctrine and operational concepts, trials can be held against realistic war-like milieu. Trials will thus, be holistic in nature rather than single equipment centric. The Training Command will be able to provide independent inputs as regards performance and suitability of equipment, thereby lending the whole process much needed credibility.

Draft Trial Directive should be prepared by the User Directorate, vetted by the Operations Directorate and scrutinised by the Equipment Directorate at the SHQ. Thereafter, it should be forwarded to the Training Command for finalisation. Trial directive must be comprehensive in all respects and must reflect underlying trial philosophy. Thereafter, the Training Command should prepare a detailed Trial Plan to include trial methodology and procedures to be adopted. Determination of sequence of trials requires a thorough knowledge of primary tasks for which equipment is being procured.

The trial units should forward their trial reports to the Training Command, who should then prepare final trial and evaluation report for submission to the SHQ.

Based on the report received from the Training Command, the SHQ should prepare the Staff Evaluation Report for submission to MoD, recommending selected equipment for induction into the service. 

As discussed above, it is essential that the trial and evaluation authority be independent of indenting and procurement agencies for objective and uninfluenced opinion as regards suitability of equipment, thereby imparting much needed credibility to the whole acquisition regime. However, MoD and SHQ must continue to exercise overall control and have the final authority as regards acceptance of equipment for acquisition.*****