How to troubleshoot DNS failure and updated changed IP in Route 53 (AWS)


After reboot of EC2 instance, public IP changed – while nslookup and dns was pointing to old IP, hence the website stopped working.

[root@ctrwebtest html]# nslookup ctrwebtest
Server: 192.168.0.2
Address: 192.168.0.2#53

Non-authoritative answer:
Name: ctrwebtest.ctrworld.com
Address: 34.218.60.21 ————– Old IP

[root@ctrwebtest html]#

Use the dig command to perform a lookup against the client DNS server that’s configured in the host’s /etc/resolv.conf file.


[root@ctrwebtest html]# dig ctrwebtest.ctrworld.com

; <<>> DiG 9.9.4-RedHat-9.9.4-74.amzn2.1.2 <<>> ctrwebtest.ctrworld.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4725
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ctrwebtest.ctrworld.com. IN A

;; ANSWER SECTION:
ctrwebtest.ctrworld.com. 169 IN A 34.218.60.21 —– Old IP

;; Query time: 0 msec
;; SERVER: 192.168.0.2#53(192.168.0.2)
;; WHEN: Wed Aug 31 18:46:47 UTC 2022
;; MSG SIZE rcvd: 68

If you add the +trace variable, the dig command can also perform a recursive lookup of a DNS record, as shown in the following example:

[root@ctrwebtest html]# dig +trace ctrwebtest.ctrworld.com

; <<>> DiG 9.9.4-RedHat-9.9.4-74.amzn2.1.2 <<>> +trace ctrwebtest.ctrworld.com
;; global options: +cmd
. 518400 IN NS G.ROOT-SERVERS.NET.
. 518400 IN NS H.ROOT-SERVERS.NET.
. 518400 IN NS I.ROOT-SERVERS.NET.
. 518400 IN NS J.ROOT-SERVERS.NET.
. 518400 IN NS K.ROOT-SERVERS.NET.
. 518400 IN NS L.ROOT-SERVERS.NET.
. 518400 IN NS M.ROOT-SERVERS.NET.
. 518400 IN NS A.ROOT-SERVERS.NET.
. 518400 IN NS B.ROOT-SERVERS.NET.
. 518400 IN NS C.ROOT-SERVERS.NET.
. 518400 IN NS D.ROOT-SERVERS.NET.
. 518400 IN NS E.ROOT-SERVERS.NET.
. 518400 IN NS F.ROOT-SERVERS.NET.
;; Received 239 bytes from 192.168.0.2#53(192.168.0.2) in 26 ms

com. 172800 IN NS l.gtld-servers.net.
com. 172800 IN NS m.gtld-servers.net.
com. 172800 IN NS j.gtld-servers.net.
com. 172800 IN NS e.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
com. 172800 IN NS d.gtld-servers.net.
com. 172800 IN NS h.gtld-servers.net.
com. 172800 IN NS f.gtld-servers.net.
com. 172800 IN NS c.gtld-servers.net.
com. 172800 IN NS g.gtld-servers.net.
com. 172800 IN NS i.gtld-servers.net.
com. 172800 IN NS k.gtld-servers.net.
com. 172800 IN NS a.gtld-servers.net.
com. 86400 IN DS 30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com. 86400 IN RRSIG DS 8 1 86400 20220913150000 20220831140000 20826 . R4bDq4hdjkcxxCS3hvtrd3mpkHogWQqXbi0KrvTbH0OPKdyaMOukt492 hGn710B9od8SgjfYpK2QiSCzVdv/v2PADbQzw/TgBIqldZyc3KqOdE1L vSVxX1/IqXMNqn5AiV0FOCclcmq29eSo6ayOQq2Ae/js7NEi8rYD7AvU vyOjxRqNbd8mj8bEgwbftVioQzDArLS6JBaX7xYa91z7oWbA4Ygt2R1F yCtS6A0y+OUAoXBZU22kgtQHcPEfXWJtM42KppoGtQ0dquxh4X0m++Kd nceJ7D2CWD1h1KyW7zYh5ZLzIS92spIE0s3Nv4a8eAgbKXKP0otwFxr0 SVDJQA==
;; Received 1186 bytes from 192.112.36.4#53(G.ROOT-SERVERS.NET) in 106 ms

ctrworld.com. 172800 IN NS ns-604.awsdns-11.net.
ctrworld.com. 172800 IN NS ns-457.awsdns-57.com.
ctrworld.com. 172800 IN NS ns-1029.awsdns-00.org.
ctrworld.com. 172800 IN NS ns-2045.awsdns-63.co.uk.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 – CK0Q2D6NI4I7EQH8NA30NS61O48UL8G5 NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20220904042433 20220828031433 32298 com. Q8/GURHUPmV/ykhy1kMRfOyVR4SElk51JPrRU8ZNQaqSdSS6LHMibxKH mZLhMJTZKmLIQuvOLw+6r8Y/QB/g8onzYAuFILpc7Um8gAHY6tfJhO6a LHRlbYzAAvnEUlQ4ydgsE3C6/BKRw+YmalYmQTHdtbI/nEHKWJMSWWO1 cWugKBKwXk/9roUzi1dwsbdSDuhCHkurzy0/dErTk7O0lA==
HECAGUU7L8JVT2TV893GE8E12DGIV9LH.com. 86400 IN NSEC3 1 1 0 – HECANTK7NHSCOVLVD7HEDDI4RVT6UHII NS DS RRSIG
HECAGUU7L8JVT2TV893GE8E12DGIV9LH.com. 86400 IN RRSIG NSEC3 8 2 86400 20220904052128 20220828041128 32298 com. ktC8t9pvCmB/bOf0Qr6UyuP2HJ5phLK7Q9kTZMf599yfIMsZm07XNTrY IaR+lZjIbOCvbgIEEFQJYIHBKrSPgVaRo9gyg/83XGqOyLN+xo+IFAP6 q9DptpYX1fEU3B9Zkk4SyKiGbgoGWAXOXNfOjG9JqFd97E7RpJ8POZAH UxBt2cyDCaKMZX8YA6Pok7SeBy+oZixi+Vcs1yRwmACkNw==
;; Received 754 bytes from 192.54.112.30#53(h.gtld-servers.net) in 119 ms

ctrwebtest.ctrworld.com. 300 IN A 34.218.60.21
ctrworld.com. 172800 IN NS ns-1029.awsdns-00.org.
ctrworld.com. 172800 IN NS ns-2045.awsdns-63.co.uk.
ctrworld.com. 172800 IN NS ns-457.awsdns-57.com.
ctrworld.com. 172800 IN NS ns-604.awsdns-11.net.
;; Received 205 bytes from 205.251.199.253#53(ns-2045.awsdns-63.co.uk) in 5 ms

[root@ctrwebtest html]# dig -t NSctrwebtest.ctrworld.com
;; Warning, ignoring invalid type NSctrwebtest.ctrworld.com

; <<>> DiG 9.9.4-RedHat-9.9.4-74.amzn2.1.2 <<>> -t NSctrwebtest.ctrworld.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17701
;; flags: qr rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;. IN NS

;; ANSWER SECTION:
. 300 IN NS h.root-servers.net.
. 300 IN NS i.root-servers.net.
. 300 IN NS j.root-servers.net.
. 300 IN NS k.root-servers.net.
. 300 IN NS l.root-servers.net.
. 300 IN NS m.root-servers.net.
. 300 IN NS a.root-servers.net.
. 300 IN NS b.root-servers.net.
. 300 IN NS c.root-servers.net.
. 300 IN NS d.root-servers.net.
. 300 IN NS e.root-servers.net.
. 300 IN NS f.root-servers.net.
. 300 IN NS g.root-servers.net.

;; Query time: 2 msec
;; SERVER: 192.168.0.2#53(192.168.0.2)
;; WHEN: Wed Aug 31 18:48:16 UTC 2022
;; MSG SIZE rcvd: 239

You can also perform a query that returns only the name servers, as shown in the following example:

[root@ctrwebtest html]# dig -t NS ctrwebtest.ctrworld.com

; <<>> DiG 9.9.4-RedHat-9.9.4-74.amzn2.1.2 <<>> -t NS ctrwebtest.ctrworld.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43697
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ctrwebtest.ctrworld.com. IN NS

;; AUTHORITY SECTION:
ctrworld.com. 300 IN SOA ns-604.awsdns-11.net. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400

;; Query time: 3 msec
;; SERVER: 192.168.0.2#53(192.168.0.2)
;; WHEN: Wed Aug 31 18:48:20 UTC 2022
;; MSG SIZE rcvd: 133

In the preceding example, http://www.ctrwebtest.ctrworld.com has the following four authoritative name servers:

ns-604.awsdns-11.net.

The following is example output for a query to http://www.ctrwebtest.ctrworld.com to one of its authoritative name servers (ns-604.awsdns-11.net). The server response indicates that http://www.ctrwebtest.ctrworld.com is available on 34.218.60.21:

[root@ctrwebtest html]# dig ctrwebtest.ctrworld.com @ns-604.awsdns-11.net.

; <<>> DiG 9.9.4-RedHat-9.9.4-74.amzn2.1.2 <<>> ctrwebtest.ctrworld.com @ns-604.awsdns-11.net.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12825
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ctrwebtest.ctrworld.com. IN A

;; ANSWER SECTION:
ctrwebtest.ctrworld.com. 300 IN A 34.218.60.21 ——– > Old IP

;; AUTHORITY SECTION:
ctrworld.com. 172800 IN NS ns-1029.awsdns-00.org.
ctrworld.com. 172800 IN NS ns-2045.awsdns-63.co.uk.
ctrworld.com. 172800 IN NS ns-457.awsdns-57.com.
ctrworld.com. 172800 IN NS ns-604.awsdns-11.net.

;; Query time: 20 msec
;; SERVER: 205.251.194.92#53(205.251.194.92)
;; WHEN: Wed Aug 31 18:50:31 UTC 2022
;; MSG SIZE rcvd: 205

[root@ctrwebtest html]#

On AWS console, navigate to Route 53 -> concerned domain name (for me it was – “ctrworld.com” -> check the concerned record and edit record to change IP to new IP. Website works after this change!

Published by heemasatapathy

Heema is currently working as Manager, Infrastructure and Cloud Solutions at Computer Technology Resources, Inc(Aquired by Application Software Technology, LLC) and oversees a team of Infrastructure and Cloud Experts performing complex upgrades and migrations to various cloud flavors including but not limited to OCI, AWS, Azure. She is a seasoned IT professional having worked worldwide - India, Japan, Malaysia, Singapore, Switzerland, and USA. She holds an Oracle Ace Pro status and loves to be part of the community group.

Leave a comment