For this we will assume that your domain is www.mydomain.com.
Also keep in mind that while you are a reseller, you can only go so far with completing these tasks, some of them requires our services, which we will happily provide.
If you are a dedicated server customer, then you can do all of the necessary steps listed below to properly set up private nameservers for your customers and not use any extra IPs.
Let's get started.
1. Login to your registrar panel (Godaddy, Enom, etc..)
2. Select the domain you want to use and find where you can 'Register New DNS'.
3. Put in ns1.yourdomain.com and ns2.yourdomain.com in the respective fields.
4. Put in the IP addresses (either yours or the ones provided to you by your host).
5. Hit the save/submit button.
NOTE: If you are a regular reseller, the rest is done by your host on their end, so notify them that you have done your part. If you run a dedicated server, read on.
6. Login to your WHM panel as 'root'.
7. Go to 'Reseller Center', select 'mydomain.com', click on the 'Edit Domains/Privilages' icon.
8. When the page loads, scroll down and put in the new nameservers, in this case ns1.mydomain.com and ns2.mydomain.com.
9 Save them and scroll down to 'DNS Function' and select 'Edit DNS Zone'.
10. You will notice that the new nameservers are now located in the top 2 text fields. Scroll all the way down to where it says 'Add new A entries'.
11. In the first text field put in the nameserver and in the second empty text field put in the IP address. So, it will look something like this:
ns1 14400 IN A 1.1.1.1
ns2 14400 IN A 1.1.1.2
12. Click Save and go back to the Reseller Center, and then to Edit Privilages/Nameservers for the correct user.
13. Scroll down to where you put in the private nameservers earlier. For each one, click on the "Assign an A record" button. This will open up a new window.
14. In this new window, click on the "Assign an A record" and that's it!