You aren't. It really is that simple. I don't care what your age is, as an experienced college-level educator I can assure you that you are not too old, because there are many nontraditional students on college campuses today (and likely some of those are older than you are), your life experience gives you many advantages over more traditional students, and with the growing nontraditional population many colleges have programs and services especially tailored for the nontraditional student.
I went back to school in my 30s and today I teach college. Yes, I have many traditional students in my classroom but every semester I have a large percentage of nontraditional students as well. I have studens in their late 20s as well as 30s, 40s, 50s, and up. I have students who have retired from one career and are looking to move into another. I have students whose children (or grandchildren) have left the nest so they are looking to enter a new stage in their life.
I also have many students balancing school with work and family. I have students who are the traditional age but are in nontraditional circumstances including children and family, work and military service, as well as sports and other activities. You are a unique person, but your situation is not as unique as you might think.
Today colleges recognize they have a changing student population and offer classes in a variety of formats including on campus, off campus, televised, and internet as well as a range of schedules including days, nights, weekends, and accelerated. In addition, there are now support services available for students who fall outside the traditional student role. Many financial aid and scholarship programs also exist specifically for the non traditional student.
In the end, it really comes down to your own gut feeling. Do you really want that degree? Are you ready to change your life? You can do it and there will be help and support available for you. Don't use your age as an excuse not to pursue your dream.