Oracle GoldenGate error 1033

Let us see how we can fix Oracle GoldenGate error 1033.

I see below errors in local report file dirrpt/DPPR01.rpt. This report file is the one created by only local data pump.

2014-08-03 14:14:37  INFO    OGG-01026  Rolling over remote file dirdat/pr000314.

Switching to next trail file dirdat/pr000317 at 2014-08-03 14:14:36 due to EOF, with current RBA 1999977
Opened trail file dirdat/pr000317 at 2014-08-03 14:14:36


Source Context :
  SourceModule            : [er.extrout]
  SourceID                : [/net/ap1109nap-vlan900/vol/ifarm_ports/ifarm_views/aime_oggcore_210874/oggcore/OpenSys/src/app/er/extrout.c]
  SourceFunction          : [complete_tcp_msg]
  SourceLine              : [1480]
  ThreadBacktrace         : [13] elements
                          : [/u01/ogg1/libgglog.so(CMessageContext::AddThreadContext()+0x26) [0xdfb166]]
                          : [/u01/ogg1/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x2c5) [0xdf7125]]
                          : [/u01/ogg1/libgglog.so(_MSG_ERR_ER_REMOTE_FILE_PROBLEM(CSourceContext*, char const*, char const*, CMessageFactory::MessageDisposition)+0x3d) [0xddc6a7]]
                          : [/u01/ogg1/extract(complete_tcp_msg(extract_def*)+0x523) [0x813dc43]]
                          : [/u01/ogg1/extract(flush_tcp(extract_def*, int)+0x217) [0x813e39b]]
                          : [/u01/ogg1/extract(flush_buffers(int)+0x67) [0x8138307]]
                          : [/u01/ogg1/extract(checkpoint_extract_info(chkpt_context_t*)+0x16) [0x8138326]]
                          : [/u01/ogg1/extract(checkpoint_status(short, short)+0xc5) [0x818c375]]
                          : [/u01/ogg1/extract(check_messages(chkpt_context_t*, short, char const*, bool)+0x575) [0x816c9e5]]
                          : [/u01/ogg1/extract(process_extract_loop()+0x16a4) [0x8187034]]
                          : [/u01/ogg1/extract(main+0x58e) [0x819819e]]
                          : [/lib/libc.so.6(__libc_start_main+0xdc) [0xa36e8c]]
                          : [/u01/ogg1/extract(__gxx_personality_v0+0x3ad) [0x81144e1]]

2014-08-03 14:14:42  ERROR   OGG-01033  There is a problem in network communication, a remote file problem, encryption keys for target and source do not match (if using ENCRYPT)
 or an unknown error. (Remote file used is dirdat/pr000314, reply received is Input rba past EOF for dirdat/pr000314; input rba: 1647117; EOF rba: 0).

***********************************************************************
*                   ** Run Time Statistics **                         *
***********************************************************************


Report at 2014-08-03 14:14:42 (activity since 2014-08-03 14:08:14)

Output to dirdat/pr:

From Table PRAJEETH.EMP:
       #                   inserts:    255882
       #                   updates:         0
       #                   deletes:    255881
       #                  discards:         0
From Table PRAJEETH.GGTEST:
       #                   inserts:         1
       #                   updates:         0
       #                   deletes:         0
       #                  discards:         0


Last log location read:
     FILE:      dirdat/pr000317
     SEQNO:     317
     RBA:       1646068
     TIMESTAMP: Not Available
     EOF:       YES
     READERR:   400


2014-08-03 14:14:42  ERROR   OGG-01668  PROCESS ABENDING.

CACHE OBJECT MANAGER statistics

CACHE MANAGER VM USAGE
vm current     =      0    vm anon queues =      0
vm anon in use =      0    vm file        =      0
vm used max    =      0    ==> CACHE BALANCED

On the remote site, on dirdat directory, I see that the remote trail file is having 0 size :

pwd
/u01/ogg1/dirdat
ls -ltr 
-rw-rw-rw- 1 oracle oinstall 1999859 Aug  3 14:14 pr000310
-rw-rw-rw- 1 oracle oinstall 1999991 Aug  3 14:14 pr000311
-rw-rw-rw- 1 oracle oinstall 1999914 Aug  3 14:14 pr000312
-rw-rw-rw- 1 oracle oinstall       0 Aug  3 14:14 pr000314
-rw-rw-rw- 1 oracle oinstall 1999868 Aug  3 14:14 pr000313
RESOLUTION

This error is caused by the absence of space in the filesystem where trailfile is written.

[oracle@oraten ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             3.0G  282M  2.5G  10% /
/dev/sda8             2.0G   81M  1.8G   5% /var
/dev/sda7             2.0G   38M  1.8G   3% /home
/dev/sda6             2.0G   36M  1.9G   2% /tmp
/dev/sda3             2.5G   69M  2.3G   3% /opt
/dev/sda2             4.4G  1.8G  2.4G  43% /usr
tmpfs                 1.7G   82M  1.6G   5% /dev/shm
/dev/sdb1              15G   15G     0 100% /u01
/dev/sdc1             6.0G  4.1G  1.6G  73% /u03

To give a complete list of trail files generated today (Aug 03,2014), I've listed below :

-rw-rw-rw- 1 oracle oinstall 1994819 Jul 20 12:48 dirdat/pr000201
-rw-rw-rw- 1 oracle oinstall 1999929 Aug  3 13:49 dirdat/pr000203
-rw-rw-rw- 1 oracle oinstall  966827 Aug  3 13:49 dirdat/pr000202
-rw-rw-rw- 1 oracle oinstall 1999845 Aug  3 13:49 dirdat/pr000204
-rw-rw-rw- 1 oracle oinstall 1999976 Aug  3 13:49 dirdat/pr000205
-rw-rw-rw- 1 oracle oinstall 1999904 Aug  3 13:50 dirdat/pr000206
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 13:50 dirdat/pr000207
-rw-rw-rw- 1 oracle oinstall 1999917 Aug  3 13:50 dirdat/pr000208
-rw-rw-rw- 1 oracle oinstall 1999882 Aug  3 13:50 dirdat/pr000209
-rw-rw-rw- 1 oracle oinstall 1999902 Aug  3 13:50 dirdat/pr000210
-rw-rw-rw- 1 oracle oinstall 1999965 Aug  3 13:51 dirdat/pr000211
-rw-rw-rw- 1 oracle oinstall 1999918 Aug  3 13:51 dirdat/pr000212
-rw-rw-rw- 1 oracle oinstall 1999883 Aug  3 13:51 dirdat/pr000213
-rw-rw-rw- 1 oracle oinstall 1999941 Aug  3 13:51 dirdat/pr000214
-rw-rw-rw- 1 oracle oinstall 1999938 Aug  3 13:51 dirdat/pr000215
-rw-rw-rw- 1 oracle oinstall 1999986 Aug  3 13:52 dirdat/pr000216
-rw-rw-rw- 1 oracle oinstall 1999911 Aug  3 13:52 dirdat/pr000217
-rw-rw-rw- 1 oracle oinstall 1999998 Aug  3 13:52 dirdat/pr000218
-rw-rw-rw- 1 oracle oinstall 1999988 Aug  3 13:52 dirdat/pr000219
-rw-rw-rw- 1 oracle oinstall 1999921 Aug  3 13:52 dirdat/pr000220
-rw-rw-rw- 1 oracle oinstall 1999996 Aug  3 13:53 dirdat/pr000221
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 13:53 dirdat/pr000222
-rw-rw-rw- 1 oracle oinstall 1999890 Aug  3 13:53 dirdat/pr000223
-rw-rw-rw- 1 oracle oinstall 1999971 Aug  3 13:53 dirdat/pr000224
-rw-rw-rw- 1 oracle oinstall 1999922 Aug  3 13:53 dirdat/pr000225
-rw-rw-rw- 1 oracle oinstall 1999970 Aug  3 13:54 dirdat/pr000226
-rw-rw-rw- 1 oracle oinstall 1999904 Aug  3 13:54 dirdat/pr000227
-rw-rw-rw- 1 oracle oinstall 1999890 Aug  3 13:54 dirdat/pr000228
-rw-rw-rw- 1 oracle oinstall 1999929 Aug  3 13:54 dirdat/pr000229
-rw-rw-rw- 1 oracle oinstall 1999942 Aug  3 13:54 dirdat/pr000230
-rw-rw-rw- 1 oracle oinstall 1999995 Aug  3 13:55 dirdat/pr000231
-rw-rw-rw- 1 oracle oinstall 1999904 Aug  3 13:55 dirdat/pr000232
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 13:55 dirdat/pr000233
-rw-rw-rw- 1 oracle oinstall 1999906 Aug  3 13:55 dirdat/pr000234
-rw-rw-rw- 1 oracle oinstall 1999875 Aug  3 13:55 dirdat/pr000235
-rw-rw-rw- 1 oracle oinstall 1999983 Aug  3 13:56 dirdat/pr000236
-rw-rw-rw- 1 oracle oinstall 1999908 Aug  3 13:56 dirdat/pr000237
-rw-rw-rw- 1 oracle oinstall 1999933 Aug  3 13:56 dirdat/pr000238
-rw-rw-rw- 1 oracle oinstall 1999968 Aug  3 13:56 dirdat/pr000239
-rw-rw-rw- 1 oracle oinstall 1999939 Aug  3 13:56 dirdat/pr000240
-rw-rw-rw- 1 oracle oinstall 1999915 Aug  3 13:57 dirdat/pr000241
-rw-rw-rw- 1 oracle oinstall 1999894 Aug  3 13:57 dirdat/pr000242
-rw-rw-rw- 1 oracle oinstall 1999976 Aug  3 13:57 dirdat/pr000243
-rw-rw-rw- 1 oracle oinstall 1999999 Aug  3 13:57 dirdat/pr000244
-rw-rw-rw- 1 oracle oinstall 1999853 Aug  3 13:57 dirdat/pr000245
-rw-rw-rw- 1 oracle oinstall 1999925 Aug  3 13:58 dirdat/pr000246
-rw-rw-rw- 1 oracle oinstall 1999931 Aug  3 13:58 dirdat/pr000247
-rw-rw-rw- 1 oracle oinstall 1999903 Aug  3 13:58 dirdat/pr000248
-rw-rw-rw- 1 oracle oinstall 1999942 Aug  3 13:58 dirdat/pr000249
-rw-rw-rw- 1 oracle oinstall 1999960 Aug  3 13:58 dirdat/pr000250
-rw-rw-rw- 1 oracle oinstall 1999912 Aug  3 13:59 dirdat/pr000251
-rw-rw-rw- 1 oracle oinstall 1999878 Aug  3 13:59 dirdat/pr000252
-rw-rw-rw- 1 oracle oinstall 1999931 Aug  3 13:59 dirdat/pr000253
-rw-rw-rw- 1 oracle oinstall 1999859 Aug  3 13:59 dirdat/pr000254
-rw-rw-rw- 1 oracle oinstall 1999965 Aug  3 13:59 dirdat/pr000255
-rw-rw-rw- 1 oracle oinstall 1999932 Aug  3 14:00 dirdat/pr000256
-rw-rw-rw- 1 oracle oinstall 1999922 Aug  3 14:00 dirdat/pr000257
-rw-rw-rw- 1 oracle oinstall 1999861 Aug  3 14:00 dirdat/pr000258
-rw-rw-rw- 1 oracle oinstall 1999896 Aug  3 14:00 dirdat/pr000259
-rw-rw-rw- 1 oracle oinstall 1999975 Aug  3 14:00 dirdat/pr000260
-rw-rw-rw- 1 oracle oinstall 1999908 Aug  3 14:01 dirdat/pr000261
-rw-rw-rw- 1 oracle oinstall 1999861 Aug  3 14:01 dirdat/pr000262
-rw-rw-rw- 1 oracle oinstall 1999937 Aug  3 14:01 dirdat/pr000263
-rw-rw-rw- 1 oracle oinstall 1999981 Aug  3 14:01 dirdat/pr000264
-rw-rw-rw- 1 oracle oinstall 1999956 Aug  3 14:01 dirdat/pr000265
-rw-rw-rw- 1 oracle oinstall 1999992 Aug  3 14:02 dirdat/pr000266
-rw-rw-rw- 1 oracle oinstall 1999909 Aug  3 14:02 dirdat/pr000267
-rw-rw-rw- 1 oracle oinstall 1999999 Aug  3 14:02 dirdat/pr000268
-rw-rw-rw- 1 oracle oinstall 1999906 Aug  3 14:02 dirdat/pr000269
-rw-rw-rw- 1 oracle oinstall 1999999 Aug  3 14:02 dirdat/pr000270
-rw-rw-rw- 1 oracle oinstall 1999906 Aug  3 14:03 dirdat/pr000271
-rw-rw-rw- 1 oracle oinstall 1999903 Aug  3 14:03 dirdat/pr000272
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 14:03 dirdat/pr000273
-rw-rw-rw- 1 oracle oinstall 1999969 Aug  3 14:03 dirdat/pr000274
-rw-rw-rw- 1 oracle oinstall 1999867 Aug  3 14:03 dirdat/pr000275
-rw-rw-rw- 1 oracle oinstall 1999972 Aug  3 14:04 dirdat/pr000276
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 14:04 dirdat/pr000277
-rw-rw-rw- 1 oracle oinstall 1999857 Aug  3 14:04 dirdat/pr000278
-rw-rw-rw- 1 oracle oinstall 1999955 Aug  3 14:04 dirdat/pr000279
-rw-rw-rw- 1 oracle oinstall 1999855 Aug  3 14:04 dirdat/pr000280
-rw-rw-rw- 1 oracle oinstall 1097083 Aug  3 14:05 dirdat/pr000281
-rw-rw-rw- 1 oracle oinstall 1999986 Aug  3 14:08 dirdat/pr000282
-rw-rw-rw- 1 oracle oinstall 1999845 Aug  3 14:08 dirdat/pr000283
-rw-rw-rw- 1 oracle oinstall 1999945 Aug  3 14:08 dirdat/pr000284
-rw-rw-rw- 1 oracle oinstall 1999937 Aug  3 14:09 dirdat/pr000285
-rw-rw-rw- 1 oracle oinstall 1999894 Aug  3 14:09 dirdat/pr000286
-rw-rw-rw- 1 oracle oinstall 1999979 Aug  3 14:09 dirdat/pr000287
-rw-rw-rw- 1 oracle oinstall 1999912 Aug  3 14:09 dirdat/pr000288
-rw-rw-rw- 1 oracle oinstall 1999930 Aug  3 14:09 dirdat/pr000289
-rw-rw-rw- 1 oracle oinstall 1999980 Aug  3 14:10 dirdat/pr000290
-rw-rw-rw- 1 oracle oinstall 1999957 Aug  3 14:10 dirdat/pr000291
-rw-rw-rw- 1 oracle oinstall 1999925 Aug  3 14:10 dirdat/pr000292
-rw-rw-rw- 1 oracle oinstall 1999979 Aug  3 14:10 dirdat/pr000293
-rw-rw-rw- 1 oracle oinstall 1999974 Aug  3 14:10 dirdat/pr000294
-rw-rw-rw- 1 oracle oinstall 1999910 Aug  3 14:11 dirdat/pr000295
-rw-rw-rw- 1 oracle oinstall 1999970 Aug  3 14:11 dirdat/pr000296
-rw-rw-rw- 1 oracle oinstall 1999940 Aug  3 14:11 dirdat/pr000297
-rw-rw-rw- 1 oracle oinstall 1999868 Aug  3 14:11 dirdat/pr000298
-rw-rw-rw- 1 oracle oinstall 1999967 Aug  3 14:11 dirdat/pr000299
-rw-rw-rw- 1 oracle oinstall 1999939 Aug  3 14:12 dirdat/pr000300
-rw-rw-rw- 1 oracle oinstall 1999948 Aug  3 14:12 dirdat/pr000301
-rw-rw-rw- 1 oracle oinstall 1999903 Aug  3 14:12 dirdat/pr000302
-rw-rw-rw- 1 oracle oinstall 1999995 Aug  3 14:12 dirdat/pr000303
-rw-rw-rw- 1 oracle oinstall 1999969 Aug  3 14:13 dirdat/pr000304
-rw-rw-rw- 1 oracle oinstall 1999941 Aug  3 14:13 dirdat/pr000305
-rw-rw-rw- 1 oracle oinstall 1999865 Aug  3 14:13 dirdat/pr000306
-rw-rw-rw- 1 oracle oinstall 1999939 Aug  3 14:13 dirdat/pr000307
-rw-rw-rw- 1 oracle oinstall 1999982 Aug  3 14:13 dirdat/pr000308
-rw-rw-rw- 1 oracle oinstall 1999919 Aug  3 14:13 dirdat/pr000309
-rw-rw-rw- 1 oracle oinstall 1999859 Aug  3 14:14 dirdat/pr000310
-rw-rw-rw- 1 oracle oinstall 1999991 Aug  3 14:14 dirdat/pr000311
-rw-rw-rw- 1 oracle oinstall 1999914 Aug  3 14:14 dirdat/pr000312
-rw-rw-rw- 1 oracle oinstall       0 Aug  3 14:14 dirdat/pr000314
-rw-rw-rw- 1 oracle oinstall 1999868 Aug  3 14:14 dirdat/pr000313

I now connect to ggsci and alter the data pump process to start the pump from Aug 03, 2014 as below :

GGSCI (voice) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     ABENDED     DPPR01      00:00:00      04:52:15
EXTRACT     ABENDED     EXPR01      00:00:00      04:39:49
REPLICAT    ABENDED     RPPR01      00:04:18      04:48:01


GGSCI (voice) 2> alter extract DPPR01, BEGIN 2014-08-03

Now on the target side, see the effect :

-rw-rw-rw- 1 oracle oinstall 1994819 Jul 20 12:48 dirdat/pr000201
-rw-rw-rw- 1 oracle oinstall 1999929 Aug  3 13:49 dirdat/pr000203
-rw-rw-rw- 1 oracle oinstall  966827 Aug  3 13:49 dirdat/pr000202
-rw-rw-rw- 1 oracle oinstall 1999845 Aug  3 13:49 dirdat/pr000204
-rw-rw-rw- 1 oracle oinstall 1999976 Aug  3 13:49 dirdat/pr000205
-rw-rw-rw- 1 oracle oinstall 1999904 Aug  3 13:50 dirdat/pr000206
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 13:50 dirdat/pr000207
-rw-rw-rw- 1 oracle oinstall 1999917 Aug  3 13:50 dirdat/pr000208
-rw-rw-rw- 1 oracle oinstall 1999882 Aug  3 13:50 dirdat/pr000209
-rw-rw-rw- 1 oracle oinstall 1999902 Aug  3 13:50 dirdat/pr000210
-rw-rw-rw- 1 oracle oinstall 1999965 Aug  3 13:51 dirdat/pr000211
-rw-rw-rw- 1 oracle oinstall 1999918 Aug  3 13:51 dirdat/pr000212
-rw-rw-rw- 1 oracle oinstall 1999883 Aug  3 13:51 dirdat/pr000213
-rw-rw-rw- 1 oracle oinstall 1999941 Aug  3 13:51 dirdat/pr000214
-rw-rw-rw- 1 oracle oinstall 1999938 Aug  3 13:51 dirdat/pr000215
-rw-rw-rw- 1 oracle oinstall 1999986 Aug  3 13:52 dirdat/pr000216
-rw-rw-rw- 1 oracle oinstall 1999911 Aug  3 13:52 dirdat/pr000217
-rw-rw-rw- 1 oracle oinstall 1999998 Aug  3 13:52 dirdat/pr000218
-rw-rw-rw- 1 oracle oinstall 1999988 Aug  3 13:52 dirdat/pr000219
-rw-rw-rw- 1 oracle oinstall 1999921 Aug  3 13:52 dirdat/pr000220
-rw-rw-rw- 1 oracle oinstall 1999996 Aug  3 13:53 dirdat/pr000221
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 13:53 dirdat/pr000222
-rw-rw-rw- 1 oracle oinstall 1999890 Aug  3 13:53 dirdat/pr000223
-rw-rw-rw- 1 oracle oinstall 1999971 Aug  3 13:53 dirdat/pr000224
-rw-rw-rw- 1 oracle oinstall 1999922 Aug  3 13:53 dirdat/pr000225
-rw-rw-rw- 1 oracle oinstall 1999970 Aug  3 13:54 dirdat/pr000226
-rw-rw-rw- 1 oracle oinstall 1999904 Aug  3 13:54 dirdat/pr000227
-rw-rw-rw- 1 oracle oinstall 1999890 Aug  3 13:54 dirdat/pr000228
-rw-rw-rw- 1 oracle oinstall 1999929 Aug  3 13:54 dirdat/pr000229
-rw-rw-rw- 1 oracle oinstall 1999942 Aug  3 13:54 dirdat/pr000230
-rw-rw-rw- 1 oracle oinstall 1999995 Aug  3 13:55 dirdat/pr000231
-rw-rw-rw- 1 oracle oinstall 1999904 Aug  3 13:55 dirdat/pr000232
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 13:55 dirdat/pr000233
-rw-rw-rw- 1 oracle oinstall 1999906 Aug  3 13:55 dirdat/pr000234
-rw-rw-rw- 1 oracle oinstall 1999875 Aug  3 13:55 dirdat/pr000235
-rw-rw-rw- 1 oracle oinstall 1999983 Aug  3 13:56 dirdat/pr000236
-rw-rw-rw- 1 oracle oinstall 1999908 Aug  3 13:56 dirdat/pr000237
-rw-rw-rw- 1 oracle oinstall 1999933 Aug  3 13:56 dirdat/pr000238
-rw-rw-rw- 1 oracle oinstall 1999968 Aug  3 13:56 dirdat/pr000239
-rw-rw-rw- 1 oracle oinstall 1999939 Aug  3 13:56 dirdat/pr000240
-rw-rw-rw- 1 oracle oinstall 1999915 Aug  3 13:57 dirdat/pr000241
-rw-rw-rw- 1 oracle oinstall 1999894 Aug  3 13:57 dirdat/pr000242
-rw-rw-rw- 1 oracle oinstall 1999976 Aug  3 13:57 dirdat/pr000243
-rw-rw-rw- 1 oracle oinstall 1999999 Aug  3 13:57 dirdat/pr000244
-rw-rw-rw- 1 oracle oinstall 1999853 Aug  3 13:57 dirdat/pr000245
-rw-rw-rw- 1 oracle oinstall 1999925 Aug  3 13:58 dirdat/pr000246
-rw-rw-rw- 1 oracle oinstall 1999931 Aug  3 13:58 dirdat/pr000247
-rw-rw-rw- 1 oracle oinstall 1999903 Aug  3 13:58 dirdat/pr000248
-rw-rw-rw- 1 oracle oinstall 1999942 Aug  3 13:58 dirdat/pr000249
-rw-rw-rw- 1 oracle oinstall 1999960 Aug  3 13:58 dirdat/pr000250
-rw-rw-rw- 1 oracle oinstall 1999912 Aug  3 13:59 dirdat/pr000251
-rw-rw-rw- 1 oracle oinstall 1999878 Aug  3 13:59 dirdat/pr000252
-rw-rw-rw- 1 oracle oinstall 1999931 Aug  3 13:59 dirdat/pr000253
-rw-rw-rw- 1 oracle oinstall 1999859 Aug  3 13:59 dirdat/pr000254
-rw-rw-rw- 1 oracle oinstall 1999965 Aug  3 13:59 dirdat/pr000255
-rw-rw-rw- 1 oracle oinstall 1999932 Aug  3 14:00 dirdat/pr000256
-rw-rw-rw- 1 oracle oinstall 1999922 Aug  3 14:00 dirdat/pr000257
-rw-rw-rw- 1 oracle oinstall 1999861 Aug  3 14:00 dirdat/pr000258
-rw-rw-rw- 1 oracle oinstall 1999896 Aug  3 14:00 dirdat/pr000259
-rw-rw-rw- 1 oracle oinstall 1999975 Aug  3 14:00 dirdat/pr000260
-rw-rw-rw- 1 oracle oinstall 1999908 Aug  3 14:01 dirdat/pr000261
-rw-rw-rw- 1 oracle oinstall 1999861 Aug  3 14:01 dirdat/pr000262
-rw-rw-rw- 1 oracle oinstall 1999937 Aug  3 14:01 dirdat/pr000263
-rw-rw-rw- 1 oracle oinstall 1999981 Aug  3 14:01 dirdat/pr000264
-rw-rw-rw- 1 oracle oinstall 1999956 Aug  3 14:01 dirdat/pr000265
-rw-rw-rw- 1 oracle oinstall 1999992 Aug  3 14:02 dirdat/pr000266
-rw-rw-rw- 1 oracle oinstall 1999909 Aug  3 14:02 dirdat/pr000267
-rw-rw-rw- 1 oracle oinstall 1999999 Aug  3 14:02 dirdat/pr000268
-rw-rw-rw- 1 oracle oinstall 1999906 Aug  3 14:02 dirdat/pr000269
-rw-rw-rw- 1 oracle oinstall 1999999 Aug  3 14:02 dirdat/pr000270
-rw-rw-rw- 1 oracle oinstall 1999906 Aug  3 14:03 dirdat/pr000271
-rw-rw-rw- 1 oracle oinstall 1999903 Aug  3 14:03 dirdat/pr000272
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 14:03 dirdat/pr000273
-rw-rw-rw- 1 oracle oinstall 1999969 Aug  3 14:03 dirdat/pr000274
-rw-rw-rw- 1 oracle oinstall 1999867 Aug  3 14:03 dirdat/pr000275
-rw-rw-rw- 1 oracle oinstall 1999972 Aug  3 14:04 dirdat/pr000276
-rw-rw-rw- 1 oracle oinstall 1999993 Aug  3 14:04 dirdat/pr000277
-rw-rw-rw- 1 oracle oinstall 1999857 Aug  3 14:04 dirdat/pr000278
-rw-rw-rw- 1 oracle oinstall 1999955 Aug  3 14:04 dirdat/pr000279
-rw-rw-rw- 1 oracle oinstall 1999855 Aug  3 14:04 dirdat/pr000280
-rw-rw-rw- 1 oracle oinstall 1097083 Aug  3 14:05 dirdat/pr000281
-rw-rw-rw- 1 oracle oinstall 1999986 Aug  3 14:08 dirdat/pr000282
-rw-rw-rw- 1 oracle oinstall 1999845 Aug  3 14:08 dirdat/pr000283
-rw-rw-rw- 1 oracle oinstall 1999945 Aug  3 14:08 dirdat/pr000284
-rw-rw-rw- 1 oracle oinstall 1999937 Aug  3 14:09 dirdat/pr000285
-rw-rw-rw- 1 oracle oinstall 1999894 Aug  3 14:09 dirdat/pr000286
-rw-rw-rw- 1 oracle oinstall 1999979 Aug  3 14:09 dirdat/pr000287
-rw-rw-rw- 1 oracle oinstall 1999912 Aug  3 14:09 dirdat/pr000288
-rw-rw-rw- 1 oracle oinstall 1999930 Aug  3 14:09 dirdat/pr000289
-rw-rw-rw- 1 oracle oinstall 1999980 Aug  3 14:10 dirdat/pr000290
-rw-rw-rw- 1 oracle oinstall 1999957 Aug  3 14:10 dirdat/pr000291
-rw-rw-rw- 1 oracle oinstall 1999925 Aug  3 14:10 dirdat/pr000292
-rw-rw-rw- 1 oracle oinstall 1999979 Aug  3 14:10 dirdat/pr000293
-rw-rw-rw- 1 oracle oinstall 1999974 Aug  3 14:10 dirdat/pr000294
-rw-rw-rw- 1 oracle oinstall 1999910 Aug  3 14:11 dirdat/pr000295
-rw-rw-rw- 1 oracle oinstall 1999970 Aug  3 14:11 dirdat/pr000296
-rw-rw-rw- 1 oracle oinstall 1999940 Aug  3 14:11 dirdat/pr000297
-rw-rw-rw- 1 oracle oinstall 1999868 Aug  3 14:11 dirdat/pr000298
-rw-rw-rw- 1 oracle oinstall 1999967 Aug  3 14:11 dirdat/pr000299
-rw-rw-rw- 1 oracle oinstall 1999939 Aug  3 14:12 dirdat/pr000300
-rw-rw-rw- 1 oracle oinstall 1999948 Aug  3 14:12 dirdat/pr000301
-rw-rw-rw- 1 oracle oinstall 1999903 Aug  3 14:12 dirdat/pr000302
-rw-rw-rw- 1 oracle oinstall 1999995 Aug  3 14:12 dirdat/pr000303
-rw-rw-rw- 1 oracle oinstall 1999969 Aug  3 14:13 dirdat/pr000304
-rw-rw-rw- 1 oracle oinstall 1999941 Aug  3 14:13 dirdat/pr000305
-rw-rw-rw- 1 oracle oinstall 1999865 Aug  3 14:13 dirdat/pr000306
-rw-rw-rw- 1 oracle oinstall 1999939 Aug  3 14:13 dirdat/pr000307
-rw-rw-rw- 1 oracle oinstall 1999982 Aug  3 14:13 dirdat/pr000308
-rw-rw-rw- 1 oracle oinstall 1999919 Aug  3 14:13 dirdat/pr000309
-rw-rw-rw- 1 oracle oinstall 1999859 Aug  3 14:14 dirdat/pr000310
-rw-rw-rw- 1 oracle oinstall 1999991 Aug  3 14:14 dirdat/pr000311
-rw-rw-rw- 1 oracle oinstall 1999914 Aug  3 14:14 dirdat/pr000312
-rw-rw-rw- 1 oracle oinstall 1999868 Aug  3 14:14 dirdat/pr000313
-rw-rw-rw- 1 oracle oinstall    1480 Aug  3 19:08 dirdat/pr000314
[oracle@oraten ogg1]$

GGSCI (oraten) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     DPPR01      00:00:00      00:00:01
EXTRACT     RUNNING     EXPR01      00:00:00      00:00:10
REPLICAT    RUNNING     RPPR01      2337:11:57    00:00:01
REPLICAT    RUNNING     RPPR02      04:54:42      00:00:01


GGSCI (oraten) 2> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING
EXTRACT     RUNNING     DPPR01      00:00:00      00:00:03
EXTRACT     RUNNING     EXPR01      00:00:15      00:00:01
REPLICAT    RUNNING     RPPR01      2337:11:57    00:00:00
REPLICAT    RUNNING     RPPR02      04:54:48      00:00:00

Done

Immediately after fixing, I didnt see any data:

SQL> select * from ggtest1;

no rows selected

But, after some time :

SQL> select * from ggtest1;

B                      C
---------------------- ---------------------------------
03-AUG-14              from voice 1

Note that the ./server is Oracle GoldenGate's Server process.

ps -eaf | grep ggserr.log
./server -w 300 -p 7819-12818 -m 7809 -k -l /u01/ogg1/ggserr.log
Reference

GoldenGate Extract Abends with "Unable to lock file" Error For Trail File (Doc ID 1112506.1)

GoldenGate Pump Abended and Can Not be Restarted Due to Locked Remote Trail, WARNING OGG-01223 Oracle GoldenGate Collector for SQL Server: Unable to lock file (Doc ID 1364965.1)

GoldenGate Configuration for Stability and Recovery (Doc ID 1451514.1)

Comments