what is the function of generator lockout relays 86G-1 and 86G-2 that are used in GE generators protection system?
Well, the description of an 86 device is a "locking-out" or "lock-out" relay. In general, the 86G-1 and 86G-2 relays on the Generator Protection Panel are actuated by one of several generator protection relays--usually, 87 devices (differential current detection devices--and occasionally, others as specified by the owner, insuring agency/company, local regulations, or utility requirements.
Sometimes, numerous devices were "grouped" to actuate 86G-1 OR 86G-2 in order to provide some differentiation as to the exact cause of the trip.
When an 86G-n relay is actuated by one or more generator protective relays/devices, the device/relay which actuated the lock-out should have a "flag" or illuminated indicating light to signify which device/relay detected a serious problem which could cause damage to the generator. When the actuating device is identified, the cause should be investigated and understood BEFORE the unit is re-started.
When an 86G-n device is actuated, the generator breaker is opened (tripped), excitation is usually de-energized, and the turbine is usually tripped (emergency shutdown).
The 86G-n relays require a manual reset (they must be manually rotated to return to the reset position) in order to permit a turbine re-start--hence the "lock-out" function.
Refer to the Generator Protection Panel elementary drawings for the exact details of what devices actuate the 86G-n relays at your site.
This link shoud explain what these lockout relays are used for in a GTG.