view related tags
Negative ping times on HP blades - September 05, 2007
Rating:
Votes: 0 rating(s)
Score: 0/0

We are having a very strange issue with twenty HP BL25p G2 Blades we have recently purchased running Citrix PS 4.0 x64 and Windows 2003 Server x64

We have thirty HP BL25P G1's which are live and run just fine.

Any chance any had similar issues with these blades or have you have heard about this issue?

We have been seeing that when we ping the timings can go a bit wrong. This happens intermittently but cause all kinds of issues in Citrix.

Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time=-1ms TTL=127
Reply from 10.1.4.91: bytes=32 time<1ms TTL=127
Reply from 10.1.4.91: bytes=32 time<1ms TTL=127
Reply from 10.1.4.91: bytes=32 time<1ms TTL=127
Reply from 10.1.4.91: bytes=32 time<1ms TTL=127

Ping statistics for 10.1.4.91:
Packets: Sent = 265, Received = 265, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = -1ms, Average = 16207423ms


As you can see from the ping times I get negative results and the average is over 16 million!!!

Regards Sean Whetstone email: sean.whetstone@reed.co.uk





Regards Sean Whetstone

sean.whetstone@reed.co.uk

London.UK
RE: Negative ping times on HP blades - September 05, 2007

Guest
Rating:
Votes: 0 rating(s)
Score: 0/0

Hi Sean,

We just recently went thru this hell and I have a solution for you. AMD has a processor driver that needs to be applied to address this issue. It appears the cores get slightly out of sync and cause this issue and in general poor performance. Your servers will eventually bluescreen when the issue gets bad enough.

you can get the driver here. It's at the bottom of the page.

http://www.amd.com/us-en/Processors/TechnicalResources/0,,30_182_871_13118,00.html

AMD Athlon™ 64 X2 Dual Core Processor Driver for Windows XP and Windows Server 2003 Version (x86 and x64 exe) 1.3.2.16 - Allows the system to automatically adjust the CPU speed, voltage and power combination that match the instantaneous user performance need. Download this Setup Installation program (EXE) to automatically update all the files necessary for installation. This package is recommended for users whom desire a graphical user interface for installation. This .EXE driver is a user friendly localized software installation of the driver designed for end-users. This driver supports AMD Athlon™ 64 X2 Dual Core processors on Windows XP SP2, Windows 2003 SP1 x84 and x64 Editions.


Try this out on a test box first. I think you will be suprised how quickly this addresses your issue.


We are using the smae hardware and x64 windows 2003 that you are using deploying citrix and office apps.

Bill Foster
Sr. Systems Engineer
WellCare
bill.foster@wellcare.com
RE: Negative ping times on HP blades - September 09, 2007
Rating:
Votes: 0 rating(s)
Score: 0/0

Thanks

So far so good, this driver seems to have resolved the negative ping issue.





Regards Sean Whetstone

sean.whetstone@reed.co.uk

London.UK
RE: Negative ping times on HP blades - September 10, 2007
Rating:
Votes: 0 rating(s)
Score: 0/0

glad to hear it Sean.

Bill Foster
RE: Negative ping times on HP blades - September 12, 2007

Guest
Rating:
Votes: 0 rating(s)
Score: 0/0

Just spoke to HP support and was supplied with their official fix for the negative time ping issue - please see below:


SUPPORT COMMUNICATION - CUSTOMER ADVISORY

Document ID: c01075682

Version: 2
Advisory: (Revision) HP ProLiant Servers Using Dual-Core or More Than One Single-Core AMD Opteron Processor May Experience Incorrect Operating System Time When Running Systems That Use the System Time Stamp Counter
NOTICE: The information in this document, including products and software versions, is current as of the Release Date. This document is subject to change without notice.

Release Date: 2007-07-16

Last Updated: 2007-07-16


DESCRIPTIONDocument Version Release Date Details
2 07/16/2007 Added Sun Solaris information.
1 06/08/2007 Original Document Release.

HP ProLiant servers configured with Dual-Core or with more than one single-core AMD Opteron processor may encounter Time Stamp Counter (TSC) drift in certain conditions. The TSC is used by some operating systems as a timekeeping source. Each processor core, whether it is a single-core processor or a dual-core processor, includes a TSC. The condition where the TSC for different processor cores becomes unsynchronized is known as TSC drift.

Note : The potential for TSC drift if the proper recommendations are not applied when using AMD Opteron 200-series, Opteron 800-series, Opteron 1200-series, Opteron 2200-series and Opteron 8200-series processors is not specific to HP ProLiant servers.

Whether or not the system is affected by TSC drift depends on the specific ProLiant server generation, the number and type of AMD Opteron processors installed, the operating system, and whether the AMD PowerNow! feature is being utilized. TSC drift can result in different symptoms and behaviors based on the operating system environment, as detailed below:

Microsoft Windows Server 2003
This condition affects operations such as network communications and performance monitoring tasks that are sensitive to system time. For example, Microsoft Active Directory domain controllers can report an Unexpected Network Error (Event ID 1054) with the following description:

Event Description:
Windows cannot obtain the domain controller name for your computer network. (An unexpected network error occurred.). Group Policy processing aborted.

In addition, a negative PING time or larger than actual PING time may be returned after issuing the PING command. The negative PING time occurs because of a Time Stamp Counter drift occurring on AMD Opteron platforms which include more than one processor core.

Red Hat Enterprise Linux, SUSE Linux Enterprise Server and Sun Solaris
Earlier releases of Red Hat Enterprise Linux 4, SUSE Linux Enterprise Server 9 and Sun Solaris 10 will default to using the Time Stamp Counter as the default time source for gettimeofday() calls. When the time stamp counter is used, the server may exhibit some inconsistent timekeeping and the following symptoms may be observed:

When a command such as "date" is typed, an incorrect system time may be displayed.
The kernel may report an error similar to the following:
kernel: Your time source seems to be instable or some driver is hogging interrupts

Newer operating systems typically do not use the TSC by default if other timers are available in the system which can be used as a timekeeping source. Other available timers include the PM_Timer and the High Precision Event Timer (HPET). All HP ProLiant servers include the PM_Timer, and the latest generation of HP ProLiant servers supporting AMD Opteron 2200-series and 8200-series processors support HPET. These timers are not affected by this condition. New operating systems such as Red Hat Enterprise Linux (RHEL) 5, SUSE Linux Enterprise Server (SLES) 10, and Microsoft Windows Server 2008 (codename Longhorn) are not affected by this issue.

Note: Some applications (e.g., Microsoft SQL Server 2005) use the Time Stamp Counter even though the operating system is configured to use a different timer as the timekeeping source. To determine if a specific application uses the TSC as the timekeeping source, contact the software vendor.

SCOPE

Any HP ProLiant server configured with more than one single-core AMD Opteron processor or configured with one (or more) dual-core AMD Opteron processors running the following operating systems:

Microsoft Windows Server 2003 (any edition)
Microsoft Windows Server 2003 x64 Edition (any edition)
Red Hat Enterprise Linux 4(x86) or earlier
Red Hat Enterprise Linux 4 (AMD64/EM64T) or earlier
SUSE Linux Enterprise Server 9 32-bit (x86) or earlier
SUSE Linux Enterprise Server 9 64-bit (AMD64/EM64T) or earlier
Sun Solaris 9
Sun Solaris 10 3/05 (32/64 bit)
VMware ESX Server 2.5.4 (or earlier)

Note: VMware ESX Server 2.5.4 with the January 2007 (or later) patch is not affected. VMware ESX Server 3.0.0 (or later) uses an alternate mechanism for timekeeping and is not affected by the potential TSC drift.

Note : The issue does not affect systems with only one single-core processor installed.

The following servers are affected when running an affected operating system:

HP ProLiant BL465c Blade Server
HP ProLiant BL685c Blade Server
HP ProLiant BL25p G2 server
HP ProLiant BL45p G2 server
HP ProLiant DL145 G3 server
HP ProLiant DL385 G2 server
HP ProLiant DL585 G2 server
HP ProLiant DL365 server
HP ProLiant ML115 server

The following servers are affected ONLY when using the AMD PowerNow! feature and running an affected operating system:

ProLiant BL25p Blade Server
HP ProLiant BL45p Blade Server
HP ProLiant DL145 G2 server
HP ProLiant DL385 server
HP ProLiant DL585 server

The following operating systems are not affected by TSC drift because these operating systems do not use the TSC as a timekeeping source:

Microsoft Windows Server 2008 (codename Longhorn)
Red Hat Enterprise Linux 5 (x86)
Red Hat Enterprise Linux 5 (AMD64/EM64T)
SUSE Linux Enterprise Server 10 (x86)
SUSE Linux Enterprise Server 10 (AMD64/EM64T)
VMware ESX Server 3.0.0 (or later)

RESOLUTION

To ensure proper operation of tasks sensitive to system time, perform either of the following actions, based on the operating system environment:

Microsoft Windows Server 2003 (any edition)
Edit the BOOT.ini file and add the parameter "/usepmtimer," then reboot the server. Adding the "/usepmtimer" parameter to the BOOT.INI file configures the Windows operating system to use the PM_TIMER, rather than the Time Stamp Counter.

Note: When installing the AMD Opteron Processor with AMD PowerNow! Technology driver Version 1.3.2.16 (or later) from AMD, the BOOT.INI file will automatically be updated with the "/usepmtimer" parameter. While the driver itself does not resolve this issue, the installation process will make the necessary changes to the BOOT.INI file to prevent the issue from occurring.

Red Hat Enterprise Linux 4 or SUSE Linux
Add the boot parameter "clock=pmtmr" to the /boot/grub/menu.lst file. Adding the "clock=pmtmr" to the /boot/grub/menu.lst file configures the operating system to use the PM_TIMER, rather than the Time Stamp Counter.

Sun Solaris
If using Sun Solaris 10 3/05 apply the 1/06 (Update 1) Patch (or later). To locate the latest version of the Solaris 10 patch, click on the following Sun Microsystems URL, and click on the desired patch:

http://www.sun.com/downloads

VMware
If using VMware ESX Server 2.5.4, update to the January 2007 Patch (or later). To locate the latest version of the ESX Server 2.5.4 patch, click on the following VMware URL, and click on the desired patch.

http://www.vmware.com/download/esx/esx2_patches.html#c4317 RECEIVE PROACTIVE UPDATES : Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively via e-mail through HP Subscriber's Choice. Sign up for Subscriber's Choice at the following URL:
http://www.hp.com/go/myadvisory

SEARCH TIP : For hints on locating similar documents on HP.com, refer to the Search Tips document: http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c00638154 .
To search for additional advisories related to System Time, use the following search string:
+ProLiant +Advisory +System Time
KEYWORDS: time sync, clock, track time


Hope this helps

Warren Estermann
RE: Negative ping times on HP blades - October 05, 2007
Rating:
Votes: 0 rating(s)
Score: 0/0

Nice to know HP finally responded to the issue. I could not wait and found another solution from AMD. Thank you for the additional information.


Bill
RE: Negative ping times on HP blades - October 31, 2007
Rating:
Votes: 0 rating(s)
Score: 0/0

I have a client that is having some very strange network issues with the HP BL25P servers. The servers will not netgotiate the same network speed/duplex, (i.e. 1000/full) and keeps dropping connections. We have tried everything to resolve this and then I came across this thread. I hope this is the resolution that we have been looking for. Thanks for posting this!
Jump to: