Overview
The following is a report completed by Jillian Nickell for IS 555 usability engineering of the Educator Licensure System, which can be viewed at https://sec3.isbe.net/IWASNET/login.aspx.
[3] Details of Tasks
For this study, I chose 7 main tasks. The seventh task is divided into 3 sub tasks that are all related to finding relevant information rather than entering it (a sort of FAQ section for teachers looking to maintain their license). For this particular site, I chose not to do a mobile test yet because the app is already so difficult to read on desktop. I instead focused on desktop testing with the plan to design and test a mobile site at some point in the future after the redesign for desktop. The tasks are as follows:
b. How many PD (professional development) hours is a graduate school class worth?
c. How can you register your license or apply for a new one?
This report references the ELIS interface. Screenshots are included throughout the study.
- Log in to your account
- Determine how many certifications you already have.
- Find the expiration date of the visual arts endorsement.
- Register your license in another region.
- Add three professional development hours to your career and technical educator license.
- How many professional development hours does the user need to keep the Visual Arts certification current?
- Determine if there is a section on the site where you can discover:
b. How many PD (professional development) hours is a graduate school class worth?
c. How can you register your license or apply for a new one?
This report references the ELIS interface. Screenshots are included throughout the study.
[1-2] Profile & background
- ID: User_1
- Age: 33
- Gender: Female
- Location: Champaign
- Occupation: Librarian
- Hours spent online per day: 6
- Websites most visited: www.illinois.edu, www.facebook.com, mail.google.com/mail
- Test date: September 26, 2018
- Test format: Desktop, Google Chrome
- Test site: Illinois Educator Licensure System
[4-6, 8] User Actions, User Non-Actions,
Potential Reasons, Interesting Findings
Task 1: Log in to your account.
The user began on this page:
The user began on this page:
The user first clicked on the "Teachers" heading along the top of the page. She reasoned that if you are a teacher looking to renew/update your license, that this is probably where she should go first, and it stood out to her.
She ended up on this page, and clicked on the "Log in to ELIS" button because she said it stood out to her.
However, this link takes her right back to this page. After a moment of scanning the page, she notices the "Educator Access" heading (circled). She notes that having three different headings that are not clearly delineated with a button or call to action to some sort was very confusing.
This brings her to this page. She clearly notices the login but also notes that this looks like a transition from an old site to a new site, which is accurate.
When the user is taken to this page, she has to search for a moment and hit "continue". She notes her irritation with being forced to click yet again instead of being delivered to the page she wanted.
The user finally arrives to the landing page, noting that it was clunky and difficult to reach/find.
Task 2: Determine how many certifications you already have.
The user began on this page:
The user began on this page:
The user notes that from the "credential center", it looks like there are three credentials. She saw three main rows and several subrows below them. The user checked the top navigation row and saw "Educator" and clicked on it. When she hovered over it, she noticed that it said "Credentials - 8", so then she wondered if there were 8 credentials instead of three. She clicks on it and goes to the next page.
From here, she could see that there are three credentials and that there is one endorsement. However, she was also immediately overwhelmed by the spreadsheet that said "registration"
She noted that while this seems like a history of registration spreadsheet, there is no sense of timeline here because dates and fiscal years are scrambled. The registration section seems to show the year the license is registered, as well as the county. They seem to be sorted by endorsement and then region; but the first column is the fiscal year, and years aren’t in any kind of year order. She said that license should be listed first then fiscal year if this is how they want to sort them, then sort by year chronologically.
She also noted that for each of the three licenses maybe instead there should be some kind of “history” pop out tab for each one instead of a complicated spreadsheet, to further organize information.
The user clicked back on "home" to return to the main spreadsheet.
She noted that they’re repeating the "licensure/approval type"(metadata) three times. She said that they should flip the hierarchy here and have the title of the license be more important.
The user also questioned: Why is this information in a spreadsheet instead of each one having its own space? Many websites have tabbed information areas or separate sections for this information.
She also noted that there are expiration dates, but they are hard to read, and that she'd like to have a warning or maybe some kind of graphic timeline showing her how much time she has.
Another question she had: If this is for me managing my license, what i need to be certified? It's not really clear from this chart.
In the end, the user determined that there are three certifications, but that the "endorsements" are hard to identify as being sub categories for each endorsement. Information is scattered across several pages and not really organized at all.
Task 3: Find the expiration date of the visual arts endorsement.
The user immediately became confused trying to find this information based on the spreadsheet on the home page.
She questioned:
"Is that endorsement directly related to the PEL certification? I can't tell from the way it's organized.
If my professional educator license had many possible endorsements, can I see a list of 200+ other endorsements I could possibly apply for? The nesting is confusing me."
She also noted:
"If an endorsement isn’t a forever thing and I need to do something to renew it, I’d like a link here to do it from the endorsement row. If there are actions to take to , why can’t I just do them here?"
The user immediately became confused trying to find this information based on the spreadsheet on the home page.
She questioned:
"Is that endorsement directly related to the PEL certification? I can't tell from the way it's organized.
If my professional educator license had many possible endorsements, can I see a list of 200+ other endorsements I could possibly apply for? The nesting is confusing me."
She also noted:
"If an endorsement isn’t a forever thing and I need to do something to renew it, I’d like a link here to do it from the endorsement row. If there are actions to take to , why can’t I just do them here?"
She then noted that the “apply for approval” tab on the right could be useful as a link from the “endorsement” or "license/aproval type" row on the spreadsheet.
The user was unable to determine the expiration for the endorsement, but did see that the PEL license that it was nested under expires in 2023. She wasn't sure what to make of this either - is the endorsement something that needs to be continually renewed as well?
The user was unable to determine the expiration for the endorsement, but did see that the PEL license that it was nested under expires in 2023. She wasn't sure what to make of this either - is the endorsement something that needs to be continually renewed as well?
Task 4: Register your license in another region.
The user noted that right hand nav is weird (see screenshot from task 3) There's a "what do you want to do" section and an "action center" section. She said the action center is weird. To her, the term "action center" makes no sense nothing about the categories listed within it looks like a link. When she hovered over a link in the action center, however, a link appears in light blue. Despite this, the links don’t take the user anywhere.
She said "I want navigation stuff to be in 'doing' space along the top, not on the right."
Finally, the user clicked on “register in another region” on the right, which took her to this page:
The user noted that right hand nav is weird (see screenshot from task 3) There's a "what do you want to do" section and an "action center" section. She said the action center is weird. To her, the term "action center" makes no sense nothing about the categories listed within it looks like a link. When she hovered over a link in the action center, however, a link appears in light blue. Despite this, the links don’t take the user anywhere.
She said "I want navigation stuff to be in 'doing' space along the top, not on the right."
Finally, the user clicked on “register in another region” on the right, which took her to this page:
The user tried to click on one of the licenses title here. Nothing happened.
She then notices the link on the right that says "register this license only". She said any kind of call to action needs to be bigger and more visible, perhaps a button. It is not clear to her that this is a link.
She then clicked on “register this license only", which took her here:
She then notices the link on the right that says "register this license only". She said any kind of call to action needs to be bigger and more visible, perhaps a button. It is not clear to her that this is a link.
She then clicked on “register this license only", which took her here:
She said that the radio button option with “save” or cancel and then a “finish” button is weird to her - it's using different terminology (save and finish in two different spots), and she doesn’t know what will happen.
When the user clicks “finish, it dumps her out on to the “educator - credentials” section. There is no clear indication that she's managed to finish registering at all.
Task 5: Add three professional development hours to your Career and Technical Educator License.
The user ends up on the “Educator” tab, thinking that there would be a place to “add more hours. At the bottom, she notes “professional development” and clicks on it.
The user ends up on the “Educator” tab, thinking that there would be a place to “add more hours. At the bottom, she notes “professional development” and clicks on it.
The user notes that boxes that say "Hours Required" and "Total Hours" along the bottom look like boxes you can edit, but you CAN’T. This is the first place she looks to add hours, but she also finds it strange that they are bound inside of a box. The user notes that the blue “click here to add professional development hours” link doesn’t make sense - there’s no information here that it will help you to reach a certain goal. There’s no “add an activity button”. Her suggestion is to add some BUTTONS that are in a consistent style to know where action points are. To her, the difference between links and action buttons are unclear.
While reading this box, she says that if there are any categories that would be fixed within the system, like qualified providers or activity names, drop downs or autofills would be great. If there isn’t , there should be a little information box with “help text” to give the user more information on this would be great. For example, she isn't sure who is a licensed provider and who isn't, and is that even a thing?
The user again notes the fact that the "cancel", "save" and "finish" radio buttons are located in two different spaces is really strange and confusing.
The user again notes the fact that the "cancel", "save" and "finish" radio buttons are located in two different spaces is really strange and confusing.
Task 6: Discover how many professional development (PD) hours the user has have to have to keep the PEL license current.
The user notes that from the spreadsheet, it is confusing because it says 45/0. She is unable to determine, from this section, what is required and what is needed. Her suggestion is more clearly marking the category.
The user notes that from the spreadsheet, it is confusing because it says 45/0. She is unable to determine, from this section, what is required and what is needed. Her suggestion is more clearly marking the category.
Task 7a: Determine if there is a place on the site to determine:
What qualifies as professional development?
The user clicks educator - highly qualified. There is no indication on this page as how to become highly qualified..
What qualifies as professional development?
The user clicks educator - highly qualified. There is no indication on this page as how to become highly qualified..
The user then clicks professional development from the drop down. This takes her to a page where she can enter professional development, but it doesn't tell her what qualifies.
The user then clicks Help - IL Licensure endorsement and approval requirements.
This takes her to a File Not Found page.
The user then clicks Help - IL Licensure endorsement and approval requirements.
This takes her to a File Not Found page.
From here, she clicks on Teachers - educator licensure - licensure - requirements.
which takes her to this page
At this point, the user has been clicking all over the site and cannot find what she is looking for easily. She becomes frustrated and gives up.
Task 7b: Determine if there is a place on the site to determine:
How many PD hours is a graduate school class worth?
The user goes to “educator - professional development" from the home screen. There is nothing here to find this information, it's just a place to enter information. After the last task, the user does not want to click through everything to find the relevant information..
How many PD hours is a graduate school class worth?
The user goes to “educator - professional development" from the home screen. There is nothing here to find this information, it's just a place to enter information. After the last task, the user does not want to click through everything to find the relevant information..
Task 7c: Determine if there is a place on the site to determine:
How can you register your new license (if you're a new teacher)
or apply for a new one?
The user clicks on "Apply for Educator License With Stipulations" under the “what do you want to do” section.
She ends up at the link below:
How can you register your new license (if you're a new teacher)
or apply for a new one?
The user clicks on "Apply for Educator License With Stipulations" under the “what do you want to do” section.
She ends up at the link below:
The user clicks the "next" button.
Finds that categories that “new license” and “apply for approval” are both confusing.
Finds that categories that “new license” and “apply for approval” are both confusing.
After clicking next, she arrives here. She notes after reading a bit that it's only for endorsements, not for a new license all around, and gives up.
The user (unsurprisingly) does not notice this path (links I clicked are in red).
Final notes/observations
The user makes several comments about the site overall after the tasks are over:
"What’s the difference between “home” and “my credentials”? Why not have all that information in one space? The action center on the right makes no sense. There are also too many pages."
The user says she wants to see who she is, first. What do I want to Do should be along the top. Determine the seven (or however many) actions that most teachers want to take, list them along the top.
The user also notes that this is not an application you don’t look at this every day. This is something you look at when you have high need or when you’ve gone above and beyond and need to get credit. Because she doesn't come here often, if something is wrong, she wants a warning message or call to action related to the license. Is it going to expire soon? If so what should she do?
The user also notes that there should be a rules and regulations page about what qualifies as professional development, how to register, etc.
The user makes several comments about the site overall after the tasks are over:
"What’s the difference between “home” and “my credentials”? Why not have all that information in one space? The action center on the right makes no sense. There are also too many pages."
The user says she wants to see who she is, first. What do I want to Do should be along the top. Determine the seven (or however many) actions that most teachers want to take, list them along the top.
The user also notes that this is not an application you don’t look at this every day. This is something you look at when you have high need or when you’ve gone above and beyond and need to get credit. Because she doesn't come here often, if something is wrong, she wants a warning message or call to action related to the license. Is it going to expire soon? If so what should she do?
The user also notes that there should be a rules and regulations page about what qualifies as professional development, how to register, etc.
[9,10 ] Relation to the readings, redesign speculations
This site, as the user noted, is not really used that often. It's a place teachers come to either update their license or register a new one. For these teachers there are really a few things they need when they visit:
These items should probably all be listed along the top somewhere.
As it stands now, the application is clunky and outdated, and information is scattered both on this application and back on the main new site. The new page is located at isbe.net. The new site has a nicer aesthetic but is still information overload and is pretty disorganized overall. Nothing is organized there either and it could probably use a card sort for navigation and informational organization. Since it was not included in the scope of the original project, I won't be able to redesign the main site (isbe.net) in addition to this application.
However, because you do get to this application via isbe.net, and it's kind of buried within the site. I usually have to google to find the link, and it's buried deep within the site. I find it silly that when you click on Teachers - log in to ELIS - that you get taken to a spreadsheet with a log in for teachers, administrators and public search. Even if these are the three options, they either shouldn't be listed here, or the call to action buttons should be bigger and more noticeable. Then you get to another "stepping stone" site, so to speak, where you have to click continue to get to the main site. So at the very least, I would recommend removing a step or two there.
I also think having an faq/rules and regulations surrounding PD hours and professional certification, and how to contact the ROE on the application is necessary. The user had lots of insights about what they thought would work which I found super helpful. There is an FAQ and a link to "contact your ROE" along the top, but it's mostly not noticeable and feels buried. Making a better top nav would probably solve this, along with removing "user info" from the top and instead moving it to its own page, like a user account, where you can change your own information, would also be helpful.
The spreadsheet where all of the user's license information is located should be taken out of spreadsheet form. Instead, this information could be organized into separate sections, perhaps tabbed or in a grid.
Deficiencies should probably be listed first and foremost, since this is urgent for users to change to keep their license current.
Graphic representations or timelines of data to make it easier to read "at a glance" would also be really useful. The spreadsheet now is not really readable.
The site can use better hierarchy and design overall. I will likely refer back to my competitive analysis of MyCarle, which shows clearly delineated sections, better use of "call to action" buttons, an "action center" on the right hand side that makes way more sense, and so one. I might also do a card sort to make better sense of the navigation. There is a lot of information that's already here that actually doesn't make a lot of sense to me, or is perhaps not categorized into a sub categories very well.
As far as relation to readings I just keep coming back to Steven Krug's "Don't Make Me Think". This site is just so old and outdated and it's obvious no one bothered to think about who was using it or how easy it might be. It definitely makes you think too hard. As Krug says,
"I should be able to "get it" - what it is and how to use it - without expending any effort thinking about it" (p 11)
This site definitely makes the user work way too hard to find what they need and do what they need.
It also makes me think of "know your user" and "task analysis" from chapter 4 of Usability Engineering. There's a lot of stuff here and not all of it is what most teachers are coming here for. Knowing the user and what they need will clarify what should be included in this application and what shouldn't. Task analysis will help with figuring out what exactly teacher's goals are and what they want to do when they come here.
- Warnings about upcoming due dates and deficiencies (probably most important on their dashboard)
- Know how many licenses they have
- How to keep their certification current/do they have the required number of PD hours
- add PD hours
- Rules and regulations surrounding certification/licensure and PD
- Registering their license or applying for a new one.
- Updating their personal profile.
These items should probably all be listed along the top somewhere.
As it stands now, the application is clunky and outdated, and information is scattered both on this application and back on the main new site. The new page is located at isbe.net. The new site has a nicer aesthetic but is still information overload and is pretty disorganized overall. Nothing is organized there either and it could probably use a card sort for navigation and informational organization. Since it was not included in the scope of the original project, I won't be able to redesign the main site (isbe.net) in addition to this application.
However, because you do get to this application via isbe.net, and it's kind of buried within the site. I usually have to google to find the link, and it's buried deep within the site. I find it silly that when you click on Teachers - log in to ELIS - that you get taken to a spreadsheet with a log in for teachers, administrators and public search. Even if these are the three options, they either shouldn't be listed here, or the call to action buttons should be bigger and more noticeable. Then you get to another "stepping stone" site, so to speak, where you have to click continue to get to the main site. So at the very least, I would recommend removing a step or two there.
I also think having an faq/rules and regulations surrounding PD hours and professional certification, and how to contact the ROE on the application is necessary. The user had lots of insights about what they thought would work which I found super helpful. There is an FAQ and a link to "contact your ROE" along the top, but it's mostly not noticeable and feels buried. Making a better top nav would probably solve this, along with removing "user info" from the top and instead moving it to its own page, like a user account, where you can change your own information, would also be helpful.
The spreadsheet where all of the user's license information is located should be taken out of spreadsheet form. Instead, this information could be organized into separate sections, perhaps tabbed or in a grid.
Deficiencies should probably be listed first and foremost, since this is urgent for users to change to keep their license current.
Graphic representations or timelines of data to make it easier to read "at a glance" would also be really useful. The spreadsheet now is not really readable.
The site can use better hierarchy and design overall. I will likely refer back to my competitive analysis of MyCarle, which shows clearly delineated sections, better use of "call to action" buttons, an "action center" on the right hand side that makes way more sense, and so one. I might also do a card sort to make better sense of the navigation. There is a lot of information that's already here that actually doesn't make a lot of sense to me, or is perhaps not categorized into a sub categories very well.
As far as relation to readings I just keep coming back to Steven Krug's "Don't Make Me Think". This site is just so old and outdated and it's obvious no one bothered to think about who was using it or how easy it might be. It definitely makes you think too hard. As Krug says,
"I should be able to "get it" - what it is and how to use it - without expending any effort thinking about it" (p 11)
This site definitely makes the user work way too hard to find what they need and do what they need.
It also makes me think of "know your user" and "task analysis" from chapter 4 of Usability Engineering. There's a lot of stuff here and not all of it is what most teachers are coming here for. Knowing the user and what they need will clarify what should be included in this application and what shouldn't. Task analysis will help with figuring out what exactly teacher's goals are and what they want to do when they come here.