Managed Server will not start when files are on the NAS - WebLogic Server - Management / Console / WLST / Configuration

We have a situation where we cannot get our server to start while the domain files are on the NAS and the file system is mounted with the lock option. We were able to start the managed server when the file system was mounted with nolock. we are running 9.2, jrockit-R27.4.0-jdk1.5.0_12 (64-bit), and Red Hat Enterprise Linux Server release 5.3.
The message below is in the admin server logs:
<Aug 6, 2009 1:56:52 PM EDT> <Error> <NodeManager> <BEA-300033> <Could not execute command "getVersion" on the
node manager. Reason: "Connection refused. Couldn't connect to NodeManager. Check that it is running at mark:5556".>
The message below is in the Node Manager logs:
<Aug 6, 2009 1:57:07 PM> <Warning> <Configuration error while reading domain directory: /usr/local/trexone/WLDomains/wesp01>
java.io.IOException: No locks available
at sun.nio.ch.FileChannelImpl.lock0(Native Method)
at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:784)
at weblogic.nodemanager.server.UserInfo.load(UserInfo.java:128)
at weblogic.nodemanager.server.DomainManager.loadUserInfo(DomainManager.java:123)
at weblogic.nodemanager.server.DomainManager.initialize(DomainManager.java:87)
at weblogic.nodemanager.server.DomainManager.<init>(DomainManager.java:52)
at weblogic.nodemanager.server.NMServer.initDomains(NMServer.java:215)
at weblogic.nodemanager.server.NMServer.start(NMServer.java:192)
at weblogic.nodemanager.server.NMServer.main(NMServer.java:329)
at weblogic.NodeManager.main(NodeManager.java:31)
Has anyone encounter this before?
When and where does the "getVersion" method function get called? 

The issue was the nfslock service was not running on the application server

Related

Nodemanager empty config file error

I am trying to use NodeManager to administer (kill, getStdOut/Err) from a managed
(clustered) server. I keep getting the message below. What is the 'Empty config
file'?
<Mar 28, 2002 4:05:48 PM PST> <Error> <NodeManager#207.170.38.155:5555> <Request:
failed to execute command 'offline' on server 'mildev04 - reason: '[NodeManagerHelper:
getSavedCommandDescriptor: Empty config file]'>
<Mar 28, 2002 4:05:48 PM PST> <Info> <NodeManager#207.170.38.155:5555> <__COMMAND_DONE__>
Is the Managed Server started via NodeManager?
If yes, have you deleted nodemanager logs while the server was running?
--
Kumar
Dan Ackerman wrote:
I am trying to use NodeManager to administer (kill, getStdOut/Err) from a managed
(clustered) server. I keep getting the message below. What is the 'Empty config
file'?
<Mar 28, 2002 4:05:48 PM PST> <Error> <NodeManager#207.170.38.155:5555> <Request:
failed to execute command 'offline' on server 'mildev04 - reason: '[NodeManagerHelper:
getSavedCommandDescriptor: Empty config file]'>
<Mar 28, 2002 4:05:48 PM PST> <Info> <NodeManager#207.170.38.155:5555> <__COMMAND_DONE__>
No, as this is the initial attempt, I'm simply starting the server(s) using startManagedWebLogic.cmd.
No, I've deleted no log files... would it be helpful to create empty log files
if they're missing? If so which ones?
____________________________________________________
Kumar Allamraju <ReplyTo#NewsGroupsOnly.com> wrote:
Is the Managed Server started via NodeManager?
If yes, have you deleted nodemanager logs while the server was running?
--
Kumar
Dan Ackerman wrote:
I am trying to use NodeManager to administer (kill, getStdOut/Err)from a managed
(clustered) server. I keep getting the message below. What is the'Empty config
file'?
<Mar 28, 2002 4:05:48 PM PST> <Error> <NodeManager#207.170.38.155:5555><Request:
failed to execute command 'offline' on server 'mildev04 - reason: '[NodeManagerHelper:
getSavedCommandDescriptor: Empty config file]'>
<Mar 28, 2002 4:05:48 PM PST> <Info> <NodeManager#207.170.38.155:5555><__COMMAND_DONE__>
Dan Ackerman wrote:
No, as this is the initial attempt, I'm simply starting the server(s) using startManagedWebLogic.cmd.You shouldn't be getting this error when you the ManagedServer was
started not via NodeManager. Here's what i get on the admin console
when i try to stop the Managed Server via admin server.
PS: the ManagedServer was started from the command line.
<Apr 8, 2002 11:44:05 AM PDT> <Info> <NodeManager> <'Stop' server
command has been invokedon server 'S1'. Please check the server state to
verify that it has been successfully stopped. For any error messages
encountered, refer to the Admin server logs.>
If the ManagedServer is not started via NodeManaged , but if you try
"Kill this server" via admin console, you will get the following error
message
<Apr 8, 2002 11:47:37 AM PDT> <Error> <NodeManager> <Could not kill
server 'S1' via Node Manager - reason: '[SecureCommandInvoker: Could not
create a socket to the NodeManager running on host '172.17.11.100:5555'
to execute command 'offline null', reason: Connection refused: connect.
Ensure that the NodeManager on host '172.17.11.100' is configured to
listen on port '5555' and that it is actively listening]'>
I tried this on 61 SP2? What ver of SP you are on?
--
Kumar
No, I've deleted no log files... would it be helpful to create empty log files
if they're missing? If so which ones?The logs will be created automatically. You don't have to create 'em
____________________________________________________
Kumar Allamraju <ReplyTo#NewsGroupsOnly.com> wrote:
Is the Managed Server started via NodeManager?
If yes, have you deleted nodemanager logs while the server was running?
--
Kumar
Dan Ackerman wrote:
I am trying to use NodeManager to administer (kill, getStdOut/Err)from a managed(clustered) server. I keep getting the message below. What is the'Empty config
file'?
<Mar 28, 2002 4:05:48 PM PST> <Error> <NodeManager#207.170.38.155:5555><Request:
failed to execute command 'offline' on server 'mildev04 - reason: '[NodeManagerHelper:
getSavedCommandDescriptor: Empty config file]'>
<Mar 28, 2002 4:05:48 PM PST> <Info> <NodeManager#207.170.38.155:5555><__COMMAND_DONE__>
>

error when starting weblogic from forthnet eclipse plugin?

Hi,
I'm getting an error when trying to start my weblogic server from the forthnet eclipse plugin. (However, it works to start the server from a command prompt on the machine it is running on, and also, it works to stop the server from eclipse)
Server and eclipse is running on Windows.
The error reported is:
<2004-sep-01 kl 16:13 CEST> <Warning> <EmbeddedLDAP> <BEA-171520> <Could not obtain an exclusive lock for directory: .\decisionsite81\ldap\ldapfiles. Waiting for 10 seconds and then retrying in case existing WebLogic Server is still shutting down.>
<2004-sep-01 kl 16:13 CEST> <Warning> <EmbeddedLDAP> <BEA-171520> <Could not obtain an exclusive lock for directory: .\decisionsite81\ldap\ldapfiles. Waiting for 10 seconds and then retrying in case existing WebLogic Server is still shutting down.>
<2004-sep-01 kl 16:13 CEST> <Warning> <EmbeddedLDAP> <BEA-171520> <Could not obtain an exclusive lock for directory: .\decisionsite81\ldap\ldapfiles. Waiting for 10 seconds and then retrying in case existing WebLogic Server is still shutting down.>
<2004-sep-01 kl 16:13 CEST> <Warning> <EmbeddedLDAP> <BEA-171520> <Could not obtain an exclusive lock for directory: .\decisionsite81\ldap\ldapfiles. Waiting for 10 seconds and then retrying in case existing WebLogic Server is still shutting down.>
<2004-sep-01 kl 16:13 CEST> <Error> <EmbeddedLDAP> <BEA-171519> <Could not obtain an exclusive lock to the embedded LDAP data files directory: .\decisionsite81\ldap\ldapfiles because another WebLogic Server is already using this directory. Ensure that the first WebLogic Server is completely shutdown and restart the server.>
<2004-sep-01 kl 16:13 CEST> <Critical> <WebLogicServer> <BEA-000364> <Server failed during initialization. Exception:weblogic.server.ServiceFailureException: Could not obtain an exclusive lock to the embedded LDAP data files directory: .\decisionsite81\ldap\ldapfiles because another WebLogic Server is already using this directory. Ensure that the first WebLogic Server is completely shutdown and restart the server.
weblogic.server.ServiceFailureException: Could not obtain an exclusive lock to the embedded LDAP data files directory: .\decisionsite81\ldap\ldapfiles because another WebLogic Server is already using this directory. Ensure that the first WebLogic Server is completely shutdown and restart the server.
     at weblogic.ldap.EmbeddedLDAP.ensureExclusiveAccess(EmbeddedLDAP.java:959)
     at weblogic.ldap.EmbeddedLDAP.initialize(EmbeddedLDAP.java:221)
     at weblogic.t3.srvr.T3Srvr.initializeHere(T3Srvr.java:814)
     at weblogic.t3.srvr.T3Srvr.initialize(T3Srvr.java:664)
     at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:342)
     at weblogic.Server.main(Server.java:32)
>
<2004-sep-01 kl 16:13 CEST> <Emergency> <WebLogicServer> <BEA-000342> <Unable to initialize the server: weblogic.server.ServiceFailureException: Could not obtain an exclusive lock to the embedded LDAP data files directory: .\decisionsite81\ldap\ldapfiles because another WebLogic Server is already using this directory. Ensure that the first WebLogic Server is completely shutdown and restart the server.>
I've tried removing the "EmbeddedLDAP.lok" file, but without luck. 
Which version of your eclipse? I am using eclipse 3.0 and I am able to start weblogic 8.1 from eclipse.
I did the following steps:
1.create a new domain through weblogic's configuration wizard --> basic weblogic server domain --> Express --> proive the user name and password --> development mode, and sun sdk (or other jdk) --> your new domain name
2. configure weblogic setting in eclipse: preferences --> weblogic --> 8.1, C:\bea\weblogic81, your newly created domain name --> C:\bea\user_projects\domains (domain directory) --> myserver --> admin (or your user name) --> password --> localhost --> 7001
after that, you should be able to click on "start weblogic server" button from toolbar.
Hope my two cents solve your problem. 
This error comes when the lock is already acquired by some other weblogic instance.So check that you have not set wls to run as windows service during install.Check in task manager.If "beasvc" process is running.Kill it and try starting the server again.It should work. 
Try removing the xxx.lok file under:
your_domain/server_name/ldap/ldapfiles
--
Rommel Sharma

starting managed server with nodemanager

Hi. I'm encountering the following error when trying to start a managed server via nodemanager. We had no problems with the same setup in Weblogic 8. Any ideas?
<Dec 13, 2006 12:39:11 AM> <INFO> <Loading domains file: D:\bea\WEBLOG~1\common\NODEMA~1\nodemanager.domains>
<Dec 13, 2006 12:39:11 AM> <INFO> <Loading identity key store: FileName=D:/bea/WEBLOG~1/server\lib\DemoIdentity.jks, Type=jks, PassPhraseUsed=true>
<Dec 13, 2006 12:39:12 AM> <INFO> <Loaded node manager configuration properties from 'D:\bea\WEBLOG~1\common\NODEMA~1\nodemanager.properties'>
<Dec 13, 2006 12:39:13 AM> <INFO> <Secure socket listener started on port 5556, host localhost>
<Dec 13, 2006 12:40:18 AM> <INFO> <Loading domains file: D:\bea\WEBLOG~1\common\NODEMA~1\nodemanager.domains>
<Dec 13, 2006 12:40:18 AM> <INFO> <Loading identity key store: FileName=D:/bea/WEBLOG~1/server\lib\DemoIdentity.jks, Type=jks, PassPhraseUsed=true>
<Dec 13, 2006 12:40:18 AM> <INFO> <Loaded node manager configuration properties from 'D:\bea\WEBLOG~1\common\NODEMA~1\nodemanager.properties'>
<Dec 13, 2006 12:40:19 AM> <INFO> <Secure socket listener started on port 5556, host localhost>
<Dec 13, 2006 12:44:58 AM> <Warning> <Exception while starting server 'managedServer_hpzd7000': java.lang.NoSuchMethodError: weblogic.nodemanager.server.ServerDir.getErrFile()Ljava/io/File;>
java.lang.NoSuchMethodError: weblogic.nodemanager.server.ServerDir.getErrFile()Ljava/io/File;
     at weblogic.nodemanager.server.ServerManager.startServer(ServerManager.java:301)
     at weblogic.nodemanager.server.ServerManager.start(ServerManager.java:248)
     at weblogic.nodemanager.server.Handler.handleStart(Handler.java:463)
     at weblogic.nodemanager.server.Handler.handleCommand(Handler.java:115)
     at weblogic.nodemanager.server.Handler.run(Handler.java:66)
     at java.lang.Thread.run(Thread.java:595) 
Hi
Can you check the Nodemanager status in the console ?
If it's fine, then, do you setup the remote start of managed servers ?
Jin 
Hi. Yes, I specified all the remote start options with good values. I also get the same error when trying to start the AdminServer with WLST:
Connecting to Node Manager ...
Successfully Connected to Node Manager.
wls:/nm/devDomain> nmStart('AdminServer')
Starting server AdminServer ...
Error Starting server AdminServer: weblogic.nodemanager.NMException: Exception while starting server 'AdminServer': java.lang.NoSuchMethodError: getErrFile
although I can start AdminServer manually....
Very strange. We'll probably look at sticking with WL 8.1, or even take a look at WebSphere. 
I have the same message when I try to start a managed server from the Admin server console. NodeManager shows the message, as does the AdminServer.log.
Working on a single machine in MS Windows XP Pro, with the configuration GUI, I created a WebLogic92 domain. I started its admin server. I started node manager. I logged onto the new admin server and then created a machine and a managed server for that machine (set to listen on port 8080).
I tried to start that managed server, NodeManager and the admin server log displayed the missing method error.
There are no weird settings specified. All of it is simple, vanilla, on one machine. In the past I succeeded with this exercise over a VPN environment, on Fedora 6 Linux, so I think I'm doing it correctly on my local Windows machine.
-Lou Mauget 
In my previous post I said I'd done this exercise OK on Fedora Linux. That made me realize that I'd started the managed server directly that time, from a script.
I tried the script for the current problem. Success. I was able to start the managed server bu using the startManagedWebLogic command. The server started OK. Afterward the Admin console properly displayed the status as RUNNING, so NodeManager returned the status OK.
Bottom line seems to be that starting a server through the NodeManager isn't working, but using a local script is working.
-Lou Mauget 
I was troubled the same problem with you.
(weblogic.nodemanager.server.ServerDir.getErrFile()Ljava/io/File)
Although i had trouble to solve this issues, finally i solved problem.
in my case, i had adopted all temporary patch provided by BEA.
ServerDir.getErrFile() is caused by CR292172_920.jar out of patchs.
ServerManager.class in CR292172_920.jar reference Server.getErrFile() but original ServerDir.class don't include getErrFile() method.
If you start managed Server using node manager in Admin Console, Node Manager can't reference ServerDir.getErrFile() method and so cause ServerDir.getErrFile().
i change ServerfDir.claass that contains getErrFile().
So, i can start Managed Server using Node Manager in Admin Console.
I think these problem has to be patched by BEA.
--
Edited by ddorigen at 02/09/2007 9:50 AM
--
Edited by ddorigen at 02/09/2007 9:51 AM 
The previous response was basically correct, albeit a little confusing to decipher. The quick fix is this -
locate WL_HOME/patch_weblogic920/patch_jars and with all servers down, rename CR292172_920.jar to something else like CR292172_920.jar.org, which will remove it from the servers' class paths. Restart admin server and node manager and try again. Mine worked just fine, but be aware that you have effectively unpatched several classes here, so be on the lookout for other things. In other words what did the patch fix in the first place???
Regards Jon 
Thank you both for posting this. Just to add some more keywords for people searching for this issue:
<BEA-300048> <Unable to start the server : Exception while starting server : java.lang.NoSuchMethodError: weblogic.nodemanager.server.ServerDir.getErrFile()Ljava/io/File;>
As soon as I mv CR292172_920.jar CR292172_920.jar.orig and restarted the admin server and nodemanager I was able to start managed servers from the admin console.
--
Edited by rrochford at 03/20/2008 6:32 AM 
I do have errors starting managed server with nodemanager. I am able to start that managed server using the script.
Error in Nodemanager.log:
<Jun 27, 2008 10:46:24 AM> <Warning> <Exception while starting server 'ManagedServer-1': java.io.IOException: Server failed to start up. See server output lo
g for more details.>
java.io.IOException: Server failed to start up. See server output log for more details.
at weblogic.nodemanager.server.ServerManager.start(ServerManager.java:296)
at weblogic.nodemanager.server.Handler.handleStart(Handler.java:511)
at weblogic.nodemanager.server.Handler.handleCommand(Handler.java:115)
at weblogic.nodemanager.server.Handler.run(Handler.java:66)
at java.lang.Thread.run(Thread.java:595)
If I see the error in managed server.out log it looks like
<Jun 26, 2008 6:11:18 PM> <Info> <NodeManager> <Server output log file is "/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/servers/ManagedServer-1/l
ogs/ManagedServer-1.out">
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: grep: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: uname: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: grep: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: uname: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: grep: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: uname: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: grep: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: uname: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: expr: not found
/RAPIDMT01/rapid01/bea/user_projects/domains/base_domain/bin/startWebLogic.sh: test: argument expected
<Jun 26, 2008 6:11:19 PM> <Info> <NodeManager> <Server failed during startup so will not be restarted>

Weblogic Managed Server FAILED_NOT_RESTARTABLE

Hi
We have installed weblogic 10.0.2.0 version on Windows 2008 server. The Node Manager service seems to be running fine however when trying to start managed server from the console fails with FAILED_NOT_RESTARTABLE message.
<boxi3snd> <Server output log file is "D:\bea\user_projects\domains\sand\servers\sand\logs\sand.out">
<Mar 28, 2011 5:38:56 PM> <INFO> <boxi3sand> <States = {sand=STARTING, AdminServer=UNKNOWN, domain_bak=UNKNOWN}>
<Mar 28, 2011 5:38:57 PM> <Info> <boxi3sand> <boxi3snd> <Server failed during startup so will not be restarted>
<Mar 28, 2011 5:38:57 PM> <Warning> <Exception while starting server 'sand': java.io.IOException: Server failed to start up. See server output log for more details.>
java.io.IOException: Server failed to start up. See server output log for more details.
     at weblogic.nodemanager.server.ServerManager.start(ServerManager.java:303)
     at weblogic.nodemanager.server.Handler.handleStart(Handler.java:542)
     at weblogic.nodemanager.server.Handler.handleCommand(Handler.java:119)
     at weblogic.nodemanager.server.Handler.run(Handler.java:66)
     at java.lang.Thread.run(Thread.java:595)
In the .out file it says that <Waiting for the process to die: 5004>
<Mar 28, 2011 5:38:57 PM> <Info> <NodeManager> <Server failed during startup so will not be restarted>
I have tried restarting the Nodemanager and Admin server but that doesnt seem to work.
Please let me know if there are other troubleshooting steps i can follow.
Thanks 
Try changing this :
goto
$DOMAIN_HOME/servers/<MANAGED SERVER NAME>/data/nodemanager/
edit the <MANAGED SERVER NAME>.state file to:
SHUTDOWN:Y:N
delete any pid file...
...
Then try restarting the server.
Edited by: supratim on Apr 9, 2011 2:23 AM 
1. Used -Xrs in "server start" at console for managed server.
2. weblogic_home\common\nodemanager\nodemanager.properties
Made changes in following parameter:
CrashRecoveryEnabled=true
StartScriptEnabled=true
3. Rename the weblogic_home\user_projects\domains\base_domain\servers\ManagedServer_1\data\nodemanager directory.
4. start Managed server from console.
5. Reboot Machine without shutting down the weblogic services
Regards.
Lam... 
We had, and still have, this problem when the PATH on the server is > about 360 characters. If ours gets up above that (and always > 400), we can't start the machine in the console.

Exception while starting server EMGC_OMS1 (upgraded to 12.1.0.2)

Hi,
I just upgraded my test environment from EM 12c R1 to R2.
Whole environment runs on Windows: Admin server and monitored targets.
After following the installation steps in the migration guide, Weblogic console starts nicely (using %DOMAIN_HOME%\startWebLogic.cmd), but I having trouble to start Enterprise Manager Cloud Control web console/server.
After trying to start EMGC_OMS1 managed server in WebLogic's console, I get the following:
- EMGC_OMS1 state goes to FAILED_NOT_RESTARTABLE.
- Admin Server's log show the following errors:
2012-Sep-28 14:33:05EMReposLoginModuleImpl: throwing LoginException: Authentication Failed: User . Password not supplied
<Sep 28, 2012 2:33:05 PM CEST> <Warning> <DeploymentService> <BEA-290014> <Invalid user name or password.>
2012-Sep-28 14:33:07EMReposLoginModuleImpl: throwing LoginException: Authentication Failed: User . Password not supplied
<Sep 28, 2012 2:33:07 PM CEST> <Error> <Configuration Management> <BEA-150035> <An attempt was made to download the configuration for >the server EMGC_OMS1 by the user with an invalid password.>
<Sep 28, 2012 2:33:13 PM CEST> <Error> <NodeManager> <BEA-300048> <Unable to start the server EMGC_OMS1 : Exception while starting server >'EMGC_OMS1'>I tried the following:
- Write again boot.properties file in %DOMAIN_HOME%\servers\EMGC_ADMINSERVER\security and %DOMAIN_HOME%\servers\EMGC_OMS1\security with the credentials for the Weblogic's admin user.
- Set username and password on "Settings for EMGC_OMS1">Configuration > Server start tab within Weblogic console.
None of these were successful.
I also compared the old (R1) installation folder with the new one for the R2, but I cannot see any relevant difference.
Do you have any suggestion on what else can be wrong?
Thanks on advance,
Angel 
Hi Angel,
Please see the following doc for instructions on starting an OMS.
http://docs.oracle.com/cd/E24628_01/doc.121/e24473/emctl.htm#BABFEEAJ
Regards,
- Loc 
Hi Loc,
thanks for your simple but effective reply.
Starting up Grid Control using the %ORACLE_HOME%/bin/emctl.bat script was the right way.
May thanks,
Angel

Categories

Resources