Random Download Latency Issue

I have Gigabit service and began having an issue about three weeks ago. At random times, my download speeds will have severe latency. Speed tests will normally show download speeds of under 2.0 MBPS, however upload speeds always remain normal at around 30-35 MBPS. Occasionally (maybe once every three days) I will lose my connection completely with my Cox supplied modem showing a red light. 

If I unplug the modem and replug it in, I return to normal speeds (500 mbps + download, 30-35 mbps upload). This fix sometimes lasts a short time, maybe 30 minutes, and sometimes much longer, up to 12+ hours. I have found no pattern to when the latency occurs, but it does seem to always need to be reset every morning when I wake up. 

At first I thought it might be a modem issue, so I swapped out my COX modem for a Surfboard 3.1, however that did not fix the problem. I have since put them Cox modem back into use. 

Cox came out to repair on 11/5. The technician checked inside and outside connections, confirming it was not an issue with my equipment or my connection at the house. He then confirmed work had been done in the neighborhood that morning to fix an issue with arching inside a box (junction box?) due to a screw touching an exposed wire. I had not had any outages since that morning, so I accepted that answer. 

I then had no issues until 11/10 when the problem reoccurred. Cox is set to come back out on 11/17 to once again try to solve the issue. They set the appointment up as a special project in the system so that they check external issues, vice going through all my connections again (which we established on 11/5 did not appear to be the issue). 

Before the first technician came out, download speeds were normally under 1.0 mbps, usually around 0.2 mbps. When the problem began to occur again on 11/5, I normally see speeds below 2.0 but well above my previous lows. This morning before I reset the modem, I saw download speeds of 5.0 mbps. Again, I have never had any issues with upload, only download latency. 

I'm of course, very frustrated. My customer service experience has actually been good thus far, the technician who came out was very thorough and really did seem to believe my issue was resolved. I just want to be assured the issue is actually resolved on the next visit.