Troubleshooting Unix/Linux Agents
In the Agents portal of the Admin Dashboard you can view the status of all Unix/Linux Agents in the system. From this dashboard, you can easily identify and troubleshoot problematic Agents in the system. You can view their operational statuses and system events in order to identify the causes and respond accordingly. For more details, see Agents and System Events.
If you are unable to troubleshoot Agents directly from the Admin Dashboard, or for some reason an Agent does not appear in the Admin Dashboard (possibly due to a connection error), then you can use the oitcons utility to check the problem.
Note the following:
-
In case of problems with Agent installation or uninstallation, you can create logs containing all the messages that were generated during the installation/uninstallation, and send them to ObserveIT support. The installation log will be stored in the directory: /etc/omonitor/<pkgname>_install.log where <pkgname> is the name of the Agent installation package. The uninstallation log will be stored in: /tmp/<pkgname>_uninstall.log.
-
If you receive an error about failed or missing dependencies during installation, you can use the -z parameter, which enables the installation script to install the missing dependencies and allow the installation to complete successfully. This parameter is supported on Linux, Ubuntu, and Debian operating systems. For example: /observeit-agent-Ubuntu-12.04-precise-5.8.0.160.run -- -i -z. For a list of library prerequisites for installing Unix/Linux Agents see Library Prerequisites for Unix/Linux Agents.
The ObserveIT Logger processes are responsible for the live recording component for each user session. When a user logs in on a Unix/Linux machine, the logger is started and begins recording the shell actions, based on a predefined data recording policy. When the user runs commands, the logger captures the output, the interception library catches the commands and the important system functions inside the commands, and forwards the information to the logger. The logger then sends the information to the ObserveIT Application Server for processing and storage.
Troubleshooting Unix/Linux Agents using the Local Console Utility "oitcons"
The oitcons utility is an ObserveIT tool for performing Unix/Linux Agent processes, such as query interception status, debugging, and logging.
You can use the oitcons utility to run some basic logger tests using the –lt switch. If necessary, the –pl switch can be used to enable and disable interception, in case the logger tests indicate a problem or indicate that a problem was fixed. When enabling or disabling Unix/Linux Agent interception, system events are generated.
The obitd service and interception module logs are stored in the obit.log file. By running oitcons -dbg, the obit.log is activated.
To run the "oitcons" utility
-
Run the command:
/opt/observeit/agent/bin/oitcons
Console usage:
oitcons <switch> [<options>]
|
Parameter |
Description |
|---|---|
|
-pl |
Preloading handler. |
|
-dbg |
Debug handler:
|
|
-service |
Start/Stop the obitd service. |
|
-register |
Register the obitd service. |
|
-unregister |
Unregister the obitd service. |
|
Example |
Run the command |
|---|---|
|
Checking or setting interception (preloading handler) |
/opt/observeit/agent/bin/oitcons -pl -h Usage: oitcons -pl <-s|-u|-c> [-h] where: -s Set observeit preloading mechanism. -u Unset observeit preloading mechanism. -c Check observeit preloading mechanism state. -h Print this usage. |
|
Checking or setting the service debug level |
/opt/observeit/agent/bin/oitcons -dbg -lvl -h Usage: oitcons -dbg -lvl [ loglevel | -u | -h] where: <no param> Display current obitd and module debug level. loglevel Set obitd and module debug level. -u Unset obitd and module debug level. -h Print this usage. loglevel parameters can have the following values:
|
|
Checking or setting the logger debug level |
/opt/observeit/agent/bin/oitcons-dbg -logger -h Usage: oitcons -dbg -logger [ -s <level> | -u <level> [<SessionId_1>] [<SessionId_2>] ... | -l | -e | -h ] where: <no param> Display the current logger debug level. -s Set the logger debug level (will affect new loggers). -u Update the debug level of all of the running loggers or the debug level of specific loggers when their SessionID's are supplied (up to 20). -l List running logger SessionIDs. -e Erase logger debug files of inactive sessions. -h Print this usage. <level> parameters can have the following values:
Note: More information about 0-7 levels may be found in the "man" of syslog (i.e., "The loglevel" folder) |
|
Starting or stopping the obitd service |
/opt/observeit/agent/bin/oitcons -service -h Usage: oitcons -service { -start | -stop | -h } [-silent] where: -h Print this usage. -start Start the obitd service. -stop Stop the obitd service. -silent Discard output messages.
|
|
Registering the Agent |
/opt/observeit/agent/bin/oitcons -register -h Usage: oitcons -register|-reg { -pkg | -s <Server> | -port <port> | -g <Policy> | -p <password> | -h } [-silent] where: -h Print this usage. -s <Server> Server to register with. -port <port> Server port. -p <password> Server password. -g <Policy> Policy ID. -pkg Called from package installation script. -silent Discard output messages. |
|
Unregistering the Agent |
/opt/observeit/agent/bin/oitcons -unregister -h Usage: oitcons -unregister|-unreg { -pkg | -s <Server> | -port <port> | -p <password> | -h } [-silent] where: -h Print this usage. -s <Server> Server to register with. -port <port> Server port. -p <password> Server password. -pkg Called from package installation script. -silent Discard output messages. |
If using a high debug log level, make sure it is used to catch only a specific action, since debug level checking can cause a lot of debug activity from many processes. The debug level check should not be used for longer than the time required to collect debug info for a specific problem or action.