vCenter Appliance WebClient down - Error: Service name is invalid

vCenter Appliance - WebClient down with "Error: Service name is invalid"


Last week I had to shutdown my lab vCenter 6.7. As it's just a lab I just powered it off instead of shutting down. After a few days I powered it back on when I wanted it back, and it booted without issues. But then WebClient wouldn't come up. Just says page cannot be displayed.

I logged in to the appliance via SSH and tried to list service status

root@vcenter [ ~ ]#  service-control --status --all
2019-04-04T10:43:41.796Z  Error: Service name "applmgmt" is invalid.
2019-04-04T10:43:41.816Z  Error: Service name "content-library" is invalid.
2019-04-04T10:43:41.837Z  Error: Service name "vcha" is invalid.
2019-04-04T10:43:41.857Z  Error: Service name "pschealth" is invalid.
2019-04-04T10:43:41.878Z  Error: Service name "sps" is invalid.
2019-04-04T10:43:41.898Z  Error: Service name "sca" is invalid.
2019-04-04T10:43:41.936Z  Error: Service name "vmcam" is invalid.
2019-04-04T10:43:41.956Z  Error: Service name "vmware-vpostgres" is invalid.
2019-04-04T10:43:41.976Z  Error: Service name "vsphere-client" is invalid.
2019-04-04T10:43:41.997Z  Error: Service name "vapi-endpoint" is invalid.
2019-04-04T10:43:42.017Z  Error: Service name "vmware-postgres-archiver" is invalid.
2019-04-04T10:43:42.037Z  Error: Service name "vsm" is invalid.
2019-04-04T10:43:42.057Z  Error: Service name "updatemgr" is invalid.
2019-04-04T10:43:42.078Z  Error: Service name "vmonapi" is invalid.
2019-04-04T10:43:42.098Z  Error: Service name "vsan-health" is invalid.
2019-04-04T10:43:42.119Z  Error: Service name "rbd" is invalid.
2019-04-04T10:43:42.156Z  Error: Service name "vpxd-svcs" is invalid.
2019-04-04T10:43:42.176Z  Error: Service name "statsmonitor" is invalid.
2019-04-04T10:43:42.264Z  Error: Service name "imagebuilder" is invalid.
2019-04-04T10:43:42.284Z  Error: Service name "vsan-dps" is invalid.
2019-04-04T10:43:42.304Z  Error: Service name "cm" is invalid.
2019-04-04T10:43:42.342Z  Error: Service name "mbcs" is invalid.
2019-04-04T10:43:42.362Z  Error: Service name "vpxd" is invalid.
2019-04-04T10:43:42.382Z  Error: Service name "netdumper" is invalid.
2019-04-04T10:43:42.403Z  Error: Service name "rhttpproxy" is invalid.
2019-04-04T10:43:42.423Z  Error: Service name "vsphere-ui" is invalid.
2019-04-04T10:43:42.444Z  Error: Service name "eam" is invalid.
2019-04-04T10:43:42.464Z  Error: Service name "perfcharts" is invalid.
2019-04-04T10:43:42.519Z  Error: Service name "analytics" is invalid.
2019-04-04T10:43:42.539Z  Error: Service name "cis-license" is invalid.
Running:
lwsmd vmafdd vmcad vmdird vmdnsd vmware-pod vmware-sts-idmd vmware-stsd vmware-vmon

Tried to start them using "service-control --start --all", but it didn't work. 

I followed below steps to get it back running.. 

Make sure that you take a backup of your vCenter before trying this (Snapshot or Clone)

1) Stop all services

root@vcenter [ ~ ]# service-control --stop --all

2) Cd to the below dir

root@vcenter [ ~ ]# cd /storage/vmware-vmon/.svcStats

3) Create a new directory and move all .Jason files there 

root@vcenter [ /storage/vmware-vmon/.svcStats ]# ls
stats_analytics.json    stats_cm.json            stats_netdumper.json   stats_sps.json            stats_vcha.json     stats_vmware-postgres-archiver.json  stats_vpxd-svcs.json    stats_vsphere-client.json
stats_applmgmt.json     stats_eam.json           stats_perfcharts.json  stats_statsmonitor.json   stats_vdcs.json     stats_vmware-sca.json                stats_vsan-dps.json     stats_vsphere-ui.json
stats_autodeploy.json   stats_imagebuilder.json  stats_pschealth.json   stats_updatemgr.json      stats_vmcam.json    stats_vmware-vpostgres.json          stats_vsan-health.json
stats_cis-license.json  stats_mbcs.json          stats_rhttpproxy.json  stats_vapi-endpoint.json  stats_vmonapi.json  stats_vpxd.json                      stats_vsm.json


root@vcenter [ /storage/vmware-vmon/.svcStats ]# mkdir StatsjsonBKP
root@vcenter [ /storage/vmware-vmon/.svcStats ]# mv *.json /storage/vmware-vmon/.svcStats/mkdir StatsjsonBKP/

4) Do another ls to confirm

root@vcenter [ /storage/vmware-vmon/.svcStats ]# ls
mkdir StatsjsonBKP

5) Now it's time to start all services

root@vcenter [ ~ ]# service-control --start --all
Operation not cancellable. Please wait for it to finish...
Performing start operation on service lwsmd...
Successfully started service lwsmd
Performing start operation on service vmafdd...
Successfully started service vmafdd
Performing start operation on service vmdird...
Successfully started service vmdird
Performing start operation on service vmcad...
Successfully started service vmcad
Performing start operation on service vmware-sts-idmd...
Successfully started service vmware-sts-idmd
Performing start operation on service vmware-stsd...
Successfully started service vmware-stsd
Performing start operation on service vmdnsd...
Successfully started service vmdnsd
Performing start operation on profile: ALL...
Successfully started profile: ALL.
Performing start operation on service vmware-pod...
Successfully started service vmware-pod

Bang!! 
Gave few mins to start web client and all good!











Comments

Post a Comment

Popular posts from this blog

vCenter Maintenance - Issue Non-ephemeral Portgroup

Image
Hi Friends, We had a issue last day due a miss configuration. We used to keep our vCenters on ephemeral port groups for troubleshooting purposes in case of any issue. We have a new vCenter now and as per standards our project team put that on a portgroup which was named as "ephemeral", but it was quiet late we found it was not actually setup as an ephemeral port group, but static instead :(. So issue was a config issue, but I wanted to share how we solved it hopping it might help someone in other situations too. The story is like this.  We planned to test an outage situation, by bringing down vCenter, to test if our VRA pre-build check can identify it. One of our lazy engineers went to Web Client of the ESXi server hosting vCenter and decided to turn the NIC off for the test. He received below error while turning it back on Description :- Reconfigure this virtual machine Virtual machine :- Non-PROD_vCenter State -: Failed - Invalid configuration for device '3
Read more

vMotion Failing at 21% with error ""The vMotion failed because the destination host did not receive data from the source host on the vMotion network. Please check your vMotion network settings and physical network configuration and ensure they are correct."

Image
vMotion Failing at 21% with error ""The vMotion failed because the destination host did not receive data from the source host on the vMotion network. Please check your vMotion network settings and physical network configuration and ensure they are correct." Scenario We built a new ESXi 6.7 Cluster and we couldn't make vMotions work there Troubleshooting steps 1) Make sure there is no IP address conflict. 2) SSH to ESXi and do a VMK Ping check For Default TCP/IP Stack vmkping 10.11.7.188 Or If you are using Multi NIC vMotion, you might want to specify which VMK interface to use vmkping -I vmk<vmkinterfacenumber> <Destination VMK IP to Ping> Eg -:  vmkping -I vmk3 192.168.1.1 For vMotion Stack I f you try above vmkping command on a ESXi host with VMK interfaces on VMK stack, they will fail with an error  Unknown interface 'vmk': Invalid argument Because the command is looking for TCP/IP stack by default and this VMK wont be listed there. So you need
Read more

applmgmt service wont start on PSC Appliace post converge operation

Scenario We had a vCenter with External PSC. We converged them and converge job was successful execpt a cert warning. After a week we tried to decommission the old PSC appliance and found that the status is shown in WebClient as "Unknown" Up on checking we found  applmgmt in stopped state. Tried to start it but it failed with below error [ ~ ]# service-control --status Running:  lwsmd pschealth vmafdd vmcad vmdird vmdnsd vmonapi vmware-analytics vmware-certificatemanagement vmware-cis-license vmware-cm vmware-rhttpproxy vmware-sca vmware-sts-idmd vmware-stsd vmware-vapi-endpoint vmware-vmon Stopped:  applmgmt vmware-statsmonitor [ ~ ]# service-control --start applmgmt Operation not cancellable. Please wait for it to finish... Performing start operation on service applmgmt... Error executing start on service applmgmt. Details {     "detail": [         {             "translatable": "An error occurred while starting service '%(0)s'",        
Read more