Re: OEM 11g Not alerting of critical status...

  • From: Guillermo Alan Bort <cicciuxdba@xxxxxxxxx>
  • To: David Lord <dlordster@xxxxxxxxx>
  • Date: Wed, 2 Mar 2011 14:59:57 -0300

Alas... it looks like we are hitting unpublished Bug 7660028 TS FULL (%)
ALERT GENERATION DOES NOT CONSIDER AVAILABLE DISK FREE SPACE.  Basically,
this Bug deals with the fact that an unpatched Database Server does not take
into account the amount of space that is on the filesystem when calculating
the Tablespace used (%) but only considers how much the datafiles that are
part of the tablespace can theoretically extend (assuming there is space on
the filesystem).
For example (assuming the filesystem is used for nothing else other than the
data file):
Tablespace TEST_TBS has 1 datafile with 2.5GB of space allocated to it
Tablespace TEST_TBS has 1.25GB of space used
The size of the filesystem that the datafile is created on is 5GB
  - 2.5GB of space is available after the tablespace is created

If the Max File Size of the datafile is set to 5GB, USED_PERCENT will be
reported correctly as 25%
If the Max File Size of the datafile is set to 10GB, USED_PERCENT will be
reported incorrectly as 12.5%.  This is because the calculation of
USED_PERCENT assumes that the datafile can extend to 10GB even though there
is only 2.5GB of disk space available and therefore the tablespace can only
extend to 5GB.  The 'Actual' USED_PERCENT should still be 25% as the
tablespace can still only extend to 5GB.

The key was in this view:
SELECT TABLESPACE_NAME TBSP_NAME
, USED_SPACE
, TABLESPACE_SIZE TBSP_SIZE
, USED_PERCENT
FROM SYS.DBA_TABLESPACE_USAGE_METRICS;

The used_percent for SYSTEM (which is currently at 94%, was about 0.5...
which makes sense since it's in a HUGE filesystem....

Isn't Oracle FUN???

Alan.-


On Wed, Mar 2, 2011 at 2:25 PM, Guillermo Alan Bort <cicciuxdba@xxxxxxxxx>wrote:

> there is nothing in dba_outstanding_alerts and the agents look like they
> are working ok. I tried removing the target and adding it back from one of
> the agents and still the same.
>
> what does dba_outstanding_alerts being empty mean? perhaps it's the
> database that's not recognizing the TBS are full...
> Alan.-
>
>
>
> On Wed, Mar 2, 2011 at 12:51 PM, David Lord <dlordster@xxxxxxxxx> wrote:
>
>> I believe that tablespace alerts are actually generated by the database -
>> OEM just picks them up and passes them on - so if the db alerting isn't
>> working, or has got out of sync with the agent you can end up with symptoms
>> like these.  It may be worthwhile checking what's showing in
>> dba_outstanding_alerts.
>>
>> If I remember correctly, there is an option in OEM to resync the agent,
>> but I can't tell you where it is as I don't have access today.  I also think
>> there's a MOS note out there somewhere for it.
>>
>> Regards
>> --
>> David Lord
>>
>> On 2 March 2011 15:02, Guillermo Alan Bort <cicciuxdba@xxxxxxxxx> wrote:
>>
>>> The thresholds are defined.
>>> job_queue_processes is set to 10 in every node.
>>>
>>> cheers
>>> Alan.-
>>>
>>>
>>> On Wed, Mar 2, 2011 at 11:48 AM, Kish Patchava 
>>> <kish.patchava@xxxxxxxxx>wrote:
>>>
>>>> Check and see if the "job_queue_processes" are greater than 0. I had a
>>>> similar issue and after changing the value to 10, everything worked.
>>>>
>>>>
>>>>
>>>> On Wed, Mar 2, 2011 at 9:35 AM, Guillermo Alan Bort <
>>>> cicciuxdba@xxxxxxxxx> wrote:
>>>>
>>>>> List,
>>>>>
>>>>>   I have an OEM 11g, with a 10.2.0.4.0 RAC target with 4 nodes. There
>>>>> are no monitoring templates configured in OEM. There is a notification 
>>>>> rule
>>>>> set for 'Database Instances' (I'm assuming this also includes cluster
>>>>> database instances, but there is a 'Cluster Database' target type possible
>>>>> in the notification rules... so I'm not sure.
>>>>>
>>>>>   We have other targets that are sending e-mail alerts and that's fine,
>>>>> but I have no record of any alert generated from this RAC. What's more, I
>>>>> tried to force an alert (set a very low threshold for a particular
>>>>> tablespace). The alert was generated in EM, but no e-mail was sent. The
>>>>> schedule is 24/7 and the email address is correct.
>>>>>
>>>>>   I am using sysman.
>>>>>
>>>>>   What is even more weird, is that I have tablespaces well over the
>>>>> warning threshold and some even past the critical threshold, and there 
>>>>> isn't
>>>>> even an alert in EM for them. I am at a loss as to even how to start to
>>>>> investigate this...
>>>>>
>>>>>   Oh, and I inherited this system and the person that set up EM si no
>>>>> longer available...
>>>>>
>>>>>    Any ideas? docs? links? magic 8 balls?
>>>>>
>>>>> thanks :-)
>>>>> Alan.-
>>>>>
>>>>
>>>>
>>>
>>
>

Other related posts: