Your attention to detail is very notable. Bring your complex carbs up some more by trying to get another meal, or two. You can bump the protein, too. If you want more fats, try to get them from mono-unsaturated, and poly-unsaturated sources. You MUST have some fat in your diet, but, you want to avoid large amounts of saturated fats, and in particular fried foods. The number one killer in America is obesity coupled with for profit medicine in treating sick folks, rather than disease prevention. Avoid the fried foods and heavily processed stuff, but, again, don't eliminate fats from your diet. Fats are caloricly dense, and that's what you need. You can also look into adding MCT oil to your diet. Glancing at your diet, it looks like you're lacking in solid complex carbs. You will not gain on just bread, or milk, or raisins. You need "stick to your rib" FOOD, like pasta, pizza, rice, oatmeal, grits, sweet potatoes, and you need MEAT, poultry, fish, whey. It wouldn't be unreasonable for you to have 600 grams of carbs, per day, if you're active. Much of that should be in the form of complex carbs, and nice and smooth on the blood sugar. Lower gylcemic index stuff is better, except near your workouts, or right before bed. For satiation, it's perfectly fine to have some simple sugar before you hit the rack. You'll also have better sports performance if you incorporate some simple sugar just before, DURING, and after your workout. Listen to your body. Keep eating. You should be able to gain about 5 to 10 pounds a month of solid weight. It's hard, for even the very best, to gain more than about 2.5 solid pounds per week. I've seen guys gain up to 80 solid pounds in as little as five months, however. Understand, your water balance can bounce your weight by as much as 10 pounds on a day to day basis. Calories are your friends. These changes take time, and patience is critical.
If you're losing with a BUNCH of calories, either you are extremely active, OR you have an undiagnosed medical disorder. Sometimes, and most commonly, that's hyper-active thyroid disease. It's an easy test.
Some sugar, pre-and-post workout, is perfectly o.k; especially post workout to get carbed back up again.
Here's another secret weapon. I can't eat the 1250kcal at each serving. I cut it in half.
When I was 17, I was 175 at 5'5", at about 12% fat. I knew nothing of nutrition, and nothing of performance enhancement, but, I did eat a lot of meat and potatoes (not fried)(I'm from The Sandhills of Nebraska). When I was 28, I was 230 at 12%. Today, my set point is around 205 to 215, which I nudge when I can, but, I just don't have the money right now to pound the calories like I'd like. Right now, I've been sitting around for a month because, after 34 years, of lifting, I had muscle tear. You are right with the 80% nutrition observation. It will serve you well. Make sure you have enough good fat in your diet. Eat good. You should be able to raise your setup as you go along.
If you STILL can't gain, then, you should get a CMP with T3/T4 test, along with a free testosterone test from a qualified health care testing facility.