ADHD or something else?!?

When my oldest son was in kindergarten (he's in 3rd grade know) I wasn't very happy with his teacher. He never had worksheets or any kind of homework that he brought home. They never did much in class either the majority of the time was spent watching movies while his teacher planned her wedding and was on the computer. Any way at the end of the year I had a meeting at the school. They were a little concerned that he was a little behind in reading. So after a lot of conversations with the school they thought it would be best to put him in a class that was for kindergarten/first/second graders that works with each child more on what ever issues there behind in or having trouble with. After meeting with the teacher of that class I decided to go ahead and have him placed in it for that coming up school year. He ended up being in the for first and second grade. During first grade they had an IEP put in place for him. I was happy that he was getting extra help that he need to hopefully catch up. I really liked his teacher and she kept him working and busy at school and always had homework for him and me to work on everyday. That's the year I really started to get concerned. There would be nights we would be at the kitchen table from the time he got home at 3:30 till 7:30-8:00 sometimes even later then that. He would get really upset either frustrated, angry, breakdown and cry because of his homework. He felt overwhelmed/frustrated/tired like his brain was on overload. There were times I would loose my cool and yell (which never solved anything and just made the situation worse) and I would feel like a piece of s#@! Like the worst mom in the world....this is my son I'm supposed to always be there for him, protect him, stand up for him and there I was yelling at him getting aggravated for something he couldn't help. So at the end of first grade we went over his IEP again and decided to keep him in the same class he was still going on to second grade though. They ended up moving his teacher and the class to a different school so he ended up having to switch schools. At the time I was a little upset about that his brother was just starting kindergarten and know they wouldn't be at the same school and he would be leaving all his friends. That year they brought up to me that they thought he might have ADHD and suggest they and my self fill out questionnaires and I could take them to his doctor to discuss it. So that's what we did I was really annoyed with how the anointment went. He didn't even really look at the papers only asked me a couple questions and then said ever so casually we can start him on meds today if you want you'll probably have to go threw a few till you find the right one and dosage that works. He told me about the side effects that would probably happen. I of course didn't put him on any meds even though the school seems to be pushing it as well. He's in third grade know and back at his old school.  What they ended up doing this year is they put him a regular 3rd grade class as his homeroom and most of his day is spent in a class next door. That is a smaller class of about 13 kids and they get extra help and work st the level thete at. Again I've had conferences with the teachers/councilor's ext and they keep trying to get me to put him on meds. Before anyone gets offended and thinks I'm putting down those that do take medication which I'm not. I know for some it truly is does help them so much  but for my son he's not hyper, aggressive, mean, violent or anything like that. He just sometimes knows something one minute and then the next has trouble with it. Not that he forgot he can't get it out. The school set up to have a neurologist come to the school to work one on one with him and find out how he learns best. They feel that they have pretty much tried everything they could think of and yet he's still at a first grade reading level. I got to meet with neurologist and I can't even tell you how happy I am to maybe finally have some answers and be able to help him. What really broke my heart and almost made me break down and cry during our meeting was when he told me he's really shock that he's held on so long and not just completely give up. He's not convinced just yet that its adhd and he told me he doesn't think he should be put on meds. I have a ton of paper work to fill out before the meetings but I can't wait to hear his recommendations  and diagnosis in the next few weeks. I will definitely share with you what I find out.I know going threw this and just truing to get answers is exhausting/frustrating . Feel free to share your story if you like.