User Testing

As each of the four test cases was completed, they were made available to the members of the steering committee and certain other users for testing. After a few of the users tried to use the first family, and found that it was too difficult to find the proper part, a set of search engines was developed (See Section 6.8.) Also, a minor error was found in the first test case model that was causing the generation to fail. The members that had been generated were removed, and the master part corrected.

The text-based search engine was released to the users at the same time as the second family of parts. These were demonstrated following a presentation to the steering committee of the first two families and the proposal for the third family. After testing the search engines and the two families, the users expressed concerns that it was too difficult to generate the parts. Several users were unable to get the parts to generate, mostly due to a requirement in CADDS5 that family of parts members be generated using a module called the Local Data Manager (LDM). Raytheon users do not normally use this module because of its extremely high overhead. The command to generate the part is long, and partially case sensitive. The LDM does not support cut-and-paste operations, so the users would have to type out the command. These concerns prompted the development of the parts server.

So far, there had been no problems with the way the parts had been created, mainly due to the extensive work done on developing a robust model, and the discussions with the committee about how the parts should be modeled. The user testing had provided significant feedback about the usage of the parts, with designers with a wide range of computer knowledge using the search engines.

The web-based search engines were released next, with a very positive reaction from several of the designers. As expected, some of the people preferred the text-based search, and others the web-based search. At the suggestion of one of the users, the scrolled list of families was added to the first page of the search engine.

The families of extrusion profiles were the next to be released. Testing of these families found the 20 character limitation on the segments of the part names, as none of the parts with the 22 character ``channel-profile-family'' segment would generate. The profiles were then renamed from ``fpts.channel-profile'' to ``fpts.profiles.channel'', for example. Because of this change, the root directory of the library became much cleaner, as the ten directories for the profile master parts and their member parts were now under one subdirectory.

Questions and comments from a few of the designers about the search engines prompted some significant rewording of the prompts. They were often confused about what the search engine was asking for. Example inputs were added to several of the prompts, and the questions were made more detailed.

The final family, the ``Creating Families of Parts'' manual (See Appendix C, the ``Using Families of Parts'' manual (See Appendix A), and the parts server (See Section 6.11) were made available at around the same time. Since there was no steering committee presentation for the fourth family due to lack of time, several individual presentations were given to members of the committee. They all said that the search engine and parts server were a big improvement over just the basic family of parts software. Although none of the designers had time before the completion of this thesis to use the ``Creating Families of Parts'' manual to create new families, several people had read over the manual, and they all said that they felt that there was sufficient information to create a new family. At the suggestion of one of the users, a table of contents with hyperlinks was added to the ``Using Families of Parts'' manual to allow users to quickly access the portion of the manual that they needed when they had a question. Links were also added, at the end of the table of contents, to the two web-based search engines. There are approximately 25 members on the CADDS5 Steering Committee, and I received feedback about the parts themselves and the search engines from almost all of the members.

Last Modified: Wed Aug 28 14:41:29 EDT 1996