This is one of the questions I get asked all the time, I am a firm supporter of using buses and trains, however, I have used a van or car on occasion especially if I am doing a trip to Jaffna for a few days. What you have to work out what is important to you, taking a car will save time and is convenient, public transport can be a hassle especially if you have a lot of luggage. the following link is from Colombo but work backwards if taking a bus or train.
How to get from Colombo to Arugam Bay?
The first thing I do is use this site FaresFare pick the highest taxi service and plot your route, that is the maximum price the trip.
Next, I would think! If the driver had one person in the car that will be Rs6000 one way, his minimum foreigner price one way, and is possible that he will have a passenger/s for return, which is not your concern unless he mentioned it and then you can haggle him lower.
So now you know the minimum. Now haggle, for me, it does not matter the number of passengers for a trip like that, same fuel, same wear and tear on the vehicle. I would haggle lower *Rs6000 over Rs3000 for the trip not per person. If the driver does a good job then tip him at the end up to the Rs6000.
Again, this is my opinion Sri Lankans ripping off foreigners is not beneficial to the tourist industry and is only a short-term fix for the country. Good news is they are plenty of good drivers that ask a fair price for a days work the key is finding them.
* A number of drivers are unregistered and do not necessarily have the correct insurance for passengers. If the driver is registered he has extra costs (insurance, license) then start at the minimum the driver can do the trip Rs6000 to the maximum from FaresFare.
Should I tip my taxi driver?