Data Tracking

This entry was posted on March 15, 2020, in Teaching Tips and tagged , , , . Bookmark the permalink. Leave a comment

And we’re back to the record keeping series! Welcome to Part 4, a post is all about data tracking for therapeutic and adaptive horseback riding programs. Again, I feel like I know so little, but what I’ve learned from the instructor’s perspective is more than enough to share, so this is my best attempt. Enjoy!

WHAT is Data Tracking

Data Tracking is the term I’ll use to describe the collecting and tracking of specific rider data over time for a later purpose.

WHY do Data Tracking

To summarize, the purpose of data tracking is to help instructors be more effective teachers and to help programs be more professional in their reporting and assessing of their program. These are the most common purposes I have seen for doing data tracking:

To Assess Rider Progress

  • Tracking the progression of a rider’s goals and objectives, riding skills, and even functional skills, can help the instructor determine how well a rider is achieving their goals.
  • If a rider is achieving every single goal, then perhaps the goals are too easy and the rider needs to be challenged more.
  • If the rider is not achieving their goals, perhaps the goals are too hard and need to be broken down into smaller steps, or perhaps this is an indicator of some other issue you need to look into (maybe the rider’s long term goals have changed, the rider is having issues at home, or their medications have changed).
  • Use this to report rider progress to their parents or caregivers at the end of the session or year.
  • Use this to explain to parents or caregivers why you made certain decisions like class pairings, horse choices, or teaching techniques.

To Assess Lesson Effectiveness

  • Tracking rider goal and skill progression, client satisfaction, and life goal achievement (and mental health survey results, if you have EAL or EFP programs) can help the instructor/professional determine the effectiveness of their lessons in a way that tracking just goal progression cannot (for example, a rider may be making very slow progress but their lessons are having a huge impact on their life, or a rider may be progressing very fast but they are not reaching their ultimate life goals).
  • The results can help the instructor or program decide if or how they can be more effective in the services they provide, such as re-evaluating teaching techniques, lesson environment, appropriateness of goals, and if the right instructor is teaching them.

To Report Program Information

  • Tracking rider data lets a program report information such as demographics, clients served, number of lessons given, types of goals made, and other information they may want to include in their marketing, fundraising, presentation, year end letters, internal program analysis, and so on.

To Report Program Impact

  • Tracking rider data such as progression and achievement of rider goals, riding skills, functional skills, as well as types of goals, client satisfaction, impact on behavior, and so on, can help a program report the impact they have on their riders for use in marketing, reporting back to donors, fundraising, and so on.

To Evaluate Programming

  • Tracking rider data such as demographics, types of goals, and client satisfaction can help a program evaluate their choices in programming, their strengths and weaknesses, if they have a niche market, if there are gaps in the program to fill with new activities, and so on.

NOT to do Scientific Research

  • In general, the data tracking I am talking about is not the same as scientific research. Yes, scientific research usually does track data, but it’s in a systematically designed study set up by qualified professionals with a hypothesis and to the end of contributing to a body of scientific knowledge.
  • It would be awesome if you included scientific research in your program, but that’s not what I’m covering in this post.

WHAT Data to Track

Above I briefly mentioned for each purpose the related data that you can track. Here is a more detailed list of those items I have seen tracked, with tips and resources. Each program is different so not all barns track all these things, these are just to give you some ideas.

WHAT TO TRACK FOR PROGRAM & INSTRUCTOR EFFECTIVENESS

Goal Progression/Achievement

  • What this is
    • Tracking the total percentage of participant goals that were met or unmet in a program.
  • Why track it
    • To assess rider progress.
    • To assess lesson effectiveness.
    • To assess program impact and the benefits of what we do
    • To report about the program in presentations and letters.
    • To report to caregivers about how the participant is progressing.
  • How to track it
    • Often a barn will collect the information at the end of each session, or at the end of the whole year, and convert the info into percentages. Ex) 75% of participants met their goals this year
  • Tips for tracking goal achievement
    • Be aware that this tracking method is very simplistic and doesn’t report the full story, so you may need to take some steps for more accurate reporting.
    • For accurate reporting, goals must be objectively measurable, such as using “SMART goals” (Specific, Measurable, Action-Oriented, Realistic, Time-Bound – for more about that click here or here).
    • Some barns include options like “progressing toward goal” and “emerging” in order to give a more accurate picture of progression, since some riders may not achieve their goal for but may still have progressed toward it.
    • Some riders have degenerative conditions so we will not see much progression, in fact, we may see maintenance or regression – which can skew results. Therefore you may want to include an option for “maintained skill” or  “regressed due to…” – I have not seen barns do this, so if you have examples, please share!
    • Consider creating a standardized goal or riding skill progression system. There can be differences in the ways instructors set their goals (for example, although all instructors know to set goals that their riders can realistically attain by the end of the session, one instructor may set harder goals than another, will affect the results). A standardized system may make results more accurate and uniform between instructors.
  • Resources
    • The Goal Attainment Scaling Method (GAS) – The GAS method was developed as a collaboration between multiple therapeutic riding centers in an attempt to find a system to measure the impact of therapeutic riding on program participants in a way that is evidence based, allows accountability, and lets results be combined across centers. This system allows you to set unique goals for each client, weights the goals to reflect importance and difficulty, pre-defines expected outcomes, uses a 5-point scale to score levels of improvement in goal attainment, uses independent raters to score achievement, and generates overall program effectiveness scores. The data you can pull from it is great: reports on progress demonstrated, the skill area focus of all the lessons, the skill area focus of different instructors, factors that limit goal attainment, and so on. I think this looks like a great method and great industry insights are coming from it. For more info about this system, check out the website linked above, this document and this PowerPoint presentation.
    • Rider Instruction, Development and Evaluation System (RIDES) – SaddleUp! developed RIDES as a simple systematic way to teach skills and set and measure goals and objectives. RIDES has 5 levels: each level has a set of unmounted and mounted components related to horsemanship skills, and for each component you can add measurable objective for your riding lesson. Once a participant meets approximately 80% of the components in one level, they move on to the next level. It is a good option for adding a standardized system of goal and skill progression to your program. The consistent format makes it easy to collect data, and while it is currently only in paper format, you can integrate it into whatever program you use for recording with their permission.

Riding & Horsemanship Skills Progression

  • What this is
    • Tracking what riding and horsemanship skills a program’s participants are working on.
  • Why track it
    • To report about the program in presentations and letters.
    • To report to caregivers about how the participant is progressing.
    • To determine what class a rider should be in.
    • To assess rider progress.
  • How to track it
    • You can easily combine this with tracking goal progression. At the end of the session when the instructors record if riders’ goals were met, include a place to categorize the goal into more general riding and horsemanship skills (such as “steering” or “trotting”, etc.) so later you can pull that data and report what percentage of riders were working on what skills this session/year.
  • Tips for tracking riding & horsemanship skill progression
    • You can use this data to create a session-end of year-end report for each rider to show or send home with their parents about what skills the rider has worked on and how they are progressing. This is easier if you use a progressive goal setting system, such as RIDES or one you make yourself.
    • Instructors can track riding skills for their own use, even if you don’t report the info to parents or your program’s stakeholders. When you look at what skills you have been working on with a rider over the year, you may notice trends that inform your future decisions. This can be as simple as using one sheet of paper to record the rider’s goals throughout the year, when they were set and met, using that to assess progress as you go or at the end of the year. EXAMPLE: My first year of instructing I looked over one of my rider’s goals for the past few sessions and saw we’d been working on “steering while in two-point” for 3 sessions! I realized I needed to break the goal down into smaller objectives, and also that he didn’t really enjoy working on that skill – so we worked hard on smaller steps, celebrated the small achievements, and moved on to something else.
    • Program Directors can use this data to determine how to best support their instructors. For example, if you notice that most of your clients are working on a particular skill (such as in the nice charts in the middle of the GAS presentation) then you can provide your instructors with resources for teaching that skill or have a group meeting to discuss ideas and troubleshooting. If you notice that some of your instructors focus on the same skills all the time, check in to see if they aren’t burnt out. If you notice that your participants are advancing in their riding skills (compared to past sessions and years), and that your instructors may not meet the requirements for teaching these advanced skills, you can use this evidence to present the case to hire another instructor who has more knowledge.
  • Resources
    • Check out the example forms on the Assessments page for ideas about tracking skill progression, such as the Participant Profile from the PATH Intl. Standards.
    • Check out the CHA book The Art of Teaching Riding, it has some good comprehensive skill tracking forms in the appendix, even one where you put all your riders on one page.

Functional Skills Progression

  • What this is
    • Tracking what functional skills a program’s participants are working on.
    • Functional skills are the physical, cognitive, and social/emotional skills that a person needs for their personal and working life (as discussed on the Assessments Post). Although adaptive and therapeutic riding instructors focus on teaching riding skills, many participants come to EAAT for its therapeutic benefits on functional skills to help them reach their life goals, so some barns include them in their goal setting.
  • Why track it
    • To report about the program in presentations and letters.
    • To discover a focus of your program that you should invest in.
    • To discover a gap in your program that you should market to.
    • To discover what your instructors need support in instructing.
    • To discover where you as an instructor need more support in teaching.
    • To assess rider progress.
  • How to track it
    • You can combine this with tracking goal progression. At the end of the session when the instructors record if riders’ goals were met, include a place to categorize the goal into its functional skill (such as “improve fine motor control” or “improve upper body strength”) so later you can pull that data and see how progress was made, or report what percentage of riders were working on what functional skills that session/year. This will be easiest if you have a pre-determined list of functional skills that you instructors can choose from.
    • You can combine this with your progress notes. For example, I have seen progress notes include a section to rate functional skills such as balance, fine motor coordination, receptive communication, and so on, on a scale of 0-5, for every lesson, with the idea that you could track progress throughout the session. In the end that method was too time-consuming so we stopped, but you could do something similar for each session.
    • You can combine this with your routine assessments. For example, you can work with a Physical Therapist or Occupational Therapist to assess the quality of a rider’s functional skills at the beginning and end of a predetermined amount of time (like the session or the year).
  • Tips for tracking functional skill progression
    • Keep in mind that anytime you work with functional skills, you should make sure you are within your realm of training. It is best to work with an Occupational/Physical/Recreational Therapist on this.
  • Resources
    • There is a list of example functional skills your barn could use about 2/3 the way down the Assessments Post, that we used in conjunction with Recreational Therapists at a barn I worked at.
    • Consider using the “FACTR-R” functional abilities assessment to base your own assessments off of. One barn I worked at had an ongoing grant that asked to see a more “scientific” method of tracking client progress. So the Recreational Therapist on staff created a functional abilities assessment for EAAT based on a common functional assessment method used in recreational therapy, the “FACTR-R”. She created an assessment that covered almost all the areas of functional ability (such as sight, vision, ambulation – with a short questionnaire for things we couldn’t assess) that was very similar to an intake lesson (like the one in the Intake II post). We did the assessment with each participant at the beginning and end of each session, and involved the students from a rec therapy class she taught at the local university. We quickly found that it took a long time to perform the assessments in general and so often, so the riders were often confused and the parents annoyed. We stopped doing this when the university class ended, but I think it would have been great to continue once or twice a year for tracking. However, we continued to use the functional abilities classifications for participant goals that she created (which I guess was “scientific” enough for the grant), which I would love to share with you but am working on getting full permission to do so.

Life Changes

  • What this is
    • The UK’s Riding for the Disabled Association (RDA) Tracker is a simple tool that measures “change” delivered through adaptive riding activities in six different areas: communication, physical changes, relationships, horsemanship, confidence and enjoyment.
  • Why track it
    • This is an option for tracking the progress and benefit of adaptive riding for your riders and your program.
    • Use it to report to riders, parents, and schools.
  • How to track it
    • It consists of a Tracker Pad which is used to record rider assessments and a simple IT program created to input the results of the assessments and create graphics to share.
  • Tips for tracking life changes
    • I have not used this program personally (I asked if I could download the program to have a look around even if I don’t have my own program, and never heard back from them), and have heard different views on it ranging from excellent to ok.
    • This is a simple concept that you could incorporate into assessments or parent questionnaires yourself, without the program.

Behavior Progression

  • What this is
    • From the Applied Behavior Analysis viewpoint, you could track a particular behavior your participant is working on, such as how many time during each lesson it occurs.
  • Why track it
    • To track improvement.
    • To learn about conditions in which the behavior occurs (many apps have a place to record what occurred before and after).
    • To gain additional data for the patient’s own behavior therapist. You can use the same app they use to track the same behavior so they can get more info from the lessons and use it to explain to the school the benefits of EAAT.
  • How to track it
    • There are behavior tracking apps you can have on your phone.
    • Teach a volunteer to use the app to track behavior during lessons.
  • Tips for behavior progression
    • I don’t know enough about this to go into more detail, but a presentation was given at the recent conference that mentioned it, and I want to include it here, as it’s different than tracking goals and skills, but rather a single behavior.

Client Satisfaction & Home Impact

  • Something else you can do to track rider progress, of sorts, is to take client satisfaction or home impact surveys, that gives you info directly from what the participant or their caregiver says. I do not have any good examples of these, but they easily could be combined with end of session assessments.

Mental Health Assessment

  • If you have an Equine Assisted Learning or Equine-Facilitated Psychotherapy program (or even a Veterans program, if it includes mental health professionals), you might use mental health assessments in your pre- and post-assessments in order to track progress in mental health over time. Again, I do not have any good examples, but if you work with a mental health professional they should know how to incorporate this.

WHAT TO TRACK FOR PROGRAM INFORMATION

Demographics

  • What this is
    • Tracking information about the participant population that you are serving.
    • Things to track include: age, town, disabilities (primary, secondary) (physical, cognitive, social/emotional), years in the program, attendance, etc.
  • Why track it
    • To report about the program in presentations and to stakeholders.
    • To discover a population that you currently are not serving.
    • To discover new program opportunities.
  • How to track it
    • This is basic information that is usually included in your participant forms already. You can pull it from that info if it’s already in the database and easily accessible, or you can have instructors include the info you want to collect with their end-of-session and goal achievement reports.

Types of Goals

  • What this is
    • The type of goals that program participants have.
    • This can be riding/horsemanship goals or functional skills goals (both already discussed above).
    • A common categorization is 1) Physical, 2) Cognitive, and 3) Social/Emotional goals.
  • Why track it
    • Same as above.
  • How to track it
    • When instructor report their students’ goals, they can also categorize the type of goal it is.

Grant Specific Information

If your program has or wants a grant that asks for specific information, then incorporate this into you data tracking process! This is barn specific. For example, I mentioned the barn I worked at whose grant needed a more “scientific” assessment process.

Horse Data

While this post is focused on tracking participant data, it bears mentioning that you can combine it with tracking horse data for your program’s internal use. For example, tracking horse usage.

HOW to do Data Tracking

Here is the basic process:

1) Collect the Data

You can collect the data from:

  • Pre-existing documents (progress notes, assessments, etc.) – set it up ahead of time to include what you need! This is easier if it’s in a database on a computer.
  • Separate documents (surveys, end-of-session forms, etc.)

2) Summarize

Put the data together into a useful format that includes a summary. This will be different depending on what you will use it for, and what program/app you are using to create it. Some examples:

  • A list
  • A pie chart
  • A graph
  • A column chart
  • A line graph

3) Use/Evaluate

  • Report the information on year end letters, quarterly/session reports, grants, your website, student reports to parents, etc.
  • Instructors use the info to discover things about and make decisions about their current riders and teaching strategies. For individual rider reports, put it in their file.
  • Programs use the info to discover strengths and weaknesses, what to support and market, new program opportunities, how to better support the instructors, etc. Save it to combine or compare with future data.

A FEW NOTES on Data Tracking

To end, I want to share a few additional thoughts about data tracking:

A Warning About Skewed Data

Be wary of factors that can skew your data and try to control for these. Factors include:

  • Instructor differences – instructors may be subjective in their reporting, may report incorrectly (their memory can be affected by waiting a long time to record data, by stress or other outside influences), instructors may differ in how easy/hard the goals are they set, and may not set goals the same way (you may want to create uniform goal setting system)
  • Data categories – maybe the words you use to describe what you are tracking are too broad, such as “trotting” when it could be “posting trot” or “sitting trot”, making it hard to combine data in a way that actually tells you something
  • Degenerative conditions – if some riders have degenerative conditions (and even old age) they may not show improvement but may decline, so tracking shows them as not reaching goals with no explanation (when really they might be doing quite well for their condition)
  • Weather – weather will affect your participants’ progress, for example, in the winter they may be stiff and unable to progress as much
  • Attendance – maybe some riders didn’t come half their lessons and therefore did not reach their goals, maybe some riders had lesson cancelled due to winter, etc.
  • Group vs. Private – I wonder if private vs group lessons affects results too…ideally they should both receive the same amount of support and be able to reach their goals, but in reality it can be hard to give everyone in a large group lesson the support they need to progress at the highest level they can because it greatly depends on your volunteers, and it can be hard to remember all the details you need to report later, and it can be hard to teach group lessons when you are a new instructor, and so on…
  • Issues – external circumstances that affect the participant’s performance such as team/volunteer issues, lesson horse issues, and so on…perhaps find a way to report these too and account for them in data reports

I don’t have any good responses to these, but if you have suggestions for how to mitigate skewed data, please share in the comments!

Coherent Industry Reporting

To more accurately report to the public what our industry does and why it is valuable, there needs to be some sort of coherent data tracking between all programs. This includes common terminology and common ways to measure progress, to name a few. PATH Intl. doesn’t advise a particular system of data tracking, and many assessment tools have no research to establish their credibility. This is an industry need that people see but are still in the beginning stages of finding a solution to.

A Word On Research

As I already mentioned, the type of data tracking I am talking about in this post is not scientific research – BUT that doesn’t mean you should forget about it! Our industry also needs research to back up what it’s doing and increase funding and grants. You can incorporate research into your program by collaborating with people who already know how to do scientific research – such as partnering with researchers or graduate students, or collaborating with related industries.

CONCLUSION

I hope that was somewhat helpful to you. As an instructor, it has been helpful for me to look back at each rider’s goals and see their progression to help me plan their future lesson objectives and plan my teaching techniques. It was also helpful to see the program’s information about what types of riders and goals we had, to see where I fit into the bigger picture. I hope you are able to incorporate a little of the above into your personal instructing, even if it’s just browsing through goals and progress notes and reflecting.

In this post I already mentioned some methods and applications for data tracking and recording things. The next post I have planned will contain all the methods I know of for

I already mentioned some methods for recording data above, but the actual hardware (programs, apps, etc.) I am going to cover in the next Record Keeping post titled…Methods! Coming soon!

Do you have anything to add? Please leave a comment!

SOURCES

****************

Note: This is not professional advice, this is a blog. I am not liable for what you do with or how you use this information. The activities explained in this blog may not be fit for every rider, riding instructor, or riding center depending on their current condition and resources. Use your best personal judgement! If you would like to contribute an activity or article, please contact me here, I would love to hear from you!

Leave a Reply

Your email address will not be published. Required fields are marked *