My Friend Loh-Ping Ying
March 11, 2017 § 4 Comments
At the time, I was working for Abe Osovsky in Yee Lee’s department. I had transferred over there from my old department where I had been developing test tools, but didn’t feel much like a tester. I wanted to do development work, but by the time I actually moved into Abe’s group, the project I had interviewed for was well past available, and someone else was working it. After a couple of months of trying to invent something for me to do, I was asked about a new project in Peter Ting’s group where a small group of people were just getting started.
That was when I met Robin (Loh-Ping) Ying. It was 1988, but I can’t recall what part of the year it was. Yat Lee was to provide Systems Engineering support, Robin and I would work the main code development, and Prem Mehotra would develop the rule-based “Expert Systems” component. Harvy Sachar provided some support as well, but he moved off to another project soon after we began.
The premise of the project was that AT&T was selling more and more private lines out to customer business locations, and in order for those lines to work on service turn-up date, the central office side of the line had to be configured correctly. Type of signalling, how would a call be initiated, what kind of equipment was at the customer site, how would numbers be passed in, was this two-way communications service, or was it one-way in (800 calling centers) or one-way out (Promotion call centers). It could also be part of a large private network (Software Defined Network) where large customers have their own dialing plan for “in-network” calls.
The problem was that more than 90% of the time, the service would not work on the turn-up date because of some error in the configuration, and it took physical people many hours, at cost to AT&T, and not the customer, to get the service working. So, corporate in Bedminster was looking for a solution, and sent Harry Schilling and John Laninen down to our location in Middletown to feed us requirements for a tool that could ask the right questions and “guide” a user through the configuration gathering process and then generate a set of “specs” for the service that could be fed downstream and perhaps decrease significantly the turn-up errors.
Harry was a tall mature man in his mid fifties who had been with the company from his youth and worked his way up into corporate. John had a similar story and both of them smoked a lot. Harry had his cigarettes, and John smoked a pipe. Both men were pretty lean body builds, and always brought an air of professionalism with them to the table. We worked when it was time to work, and if we traveled with them, we played when it was time to play. Corporate always picked up the tab on those trips!
Yat, Harvey, Robin and I sat in on the first meeting where John and Harry presented one of their early drafts of something they called the “Customer Location Profile” (CLP). Since it was paper, it had a lot of choices and a set of instructions that read something like “If your answer to question 4 was ‘Touch Tone’ then what is … If it was DTMF, then what …”. Lots of instructions about what parts of the form to fill out based on answers to “setup” style questions. The process in development was that a “technical” person would accompany the sale as support and gather the information. A team at each provisioning center would validate it based on their knowledge of the network and how these services need to be configured, and then the CLP would be attached to the order.Very human intensive.
Seemed like the perfect problem for a software tool, that could present the questions to be answered based on answers to questions already presented. Then a final validation could be performed, and if it passed, then the user could print out the CLP and send it down the pike for service implementation. For validation, that was where Prem was brought in. Prem was one of the easiest-going, gentlest, most timid people I had ever met. He hailed from India and his handshake was very soft and non-threatening. Soft-spoken and always answered his phone “Hello, this is Prem”, or when we were in conference calls and he had something to ask or say, he would preface with “Hello, this is Prem”. Everyone on the call already knew who it was, because, well, Prem! Anyway, he was the “Expert Systems” guy, and we wanted to demonstrate this technology in this tool, and he and Yat worked together formulating rules to validate the input data.
Since our department, and primarily Peter Ting’s group prided themselves in using state of the art solutions, it was very important to say that the project was using rule-based expert systems software to validate the data, and thereby produce a higher percentage of error-free provisioning data. One design problem was, that this was built into a library, that was linked in, and was designed to be a post-input form of validation. What this meant was that the user entered their data, seemingly fine, and then when they indicated they are done, the software would submit their data to the library, and it spit out a bunch of lines like “based on your answer of blah, to question 3 about this, your answer of that in question 6 is wrong and can only be this or that”. Some of the rules were a little more complicated, but the nature of the interface, was a call to the library would be made, the library would initialize itself, and then process the entire form, and produce an output of errors (if they existed), and those errors would be presented in a dialog to the user. They could print the errors out, and then go back through the screens and correct those errors, and submit the form for validation as many times as they needed to to get the data correct.
It wasn’t long before the user community, rightfully, asked “Why do I have to wait till the end to find out I made a mistake? Can’t you move that up and tell me during data entry? Better yet, don’t present me wrong options.” This was a very valid point, however at the time, there were other issues besides the nature of the expert system library, that affected this request.
This was 1988. There was no internet as we know it now, and there were no laptops, no ipads, even the nature of PC’s was pretty limited. The labs was a Unix shop, and in those days, high end user interfaces, not PC-based were expensive workstations like Sun and silicon graphics, or HP. Too expensive to propose outfitting work centers with them on a project like this. We had to present the interface within a 24 line by 80 column character terminal device. While event-based processing UIs were available on workstations (X-Windows), they hadn’t found their way into character device screens yet and that left us using a homegrown package from another project called Libforms. Libforms was based on a Murray Hill developed package called Tabs, and that was implemented on top of the curses library which used a login session’s TERM variable to make sure that well known commands were translated into the correct escape codes pertinent to that terminal device. A certain level of flexibility was available, but Libforms mainly processed data as a whole screen, though it did allow a validation function for each field after data was entered. The main point though was the screen had to be filled out, and then submitted to get the next screen.
We were constantly challenged with the problem of deciding how much data to collect on one screen before asking the user to move to the next screen. The data had some organization to it, so there were natural division lines for some screens, however we had to balance the need to drive screen content with the need to keep the number of screens limited. As the data got more technical, the number of options increased, and the complexity increased. Kind of like Chess, in that on the first move there are only so many possibilities, but by the 3rd move you have thousands. We didn’t want an approach that was simply one field per screen, as that would be too many screens, and opens the issue of indexed navigation (how do you move back and forth through that mess). Also at the time we were borrowing the services of Creighton Clark, another Peter Ting developer who was the primary maintainer of Libforms. He was open to some suggestions, but there was a limit to how much change he would consider in the library. Creighton was a very pleasant, very much more aged developer than the rest of us, that only wanted to do simple updates to the library. For example, when I asked him to make sure that instead of passing a copy of the name for a form’s field back in validation, to pass instead the actual pointer to the name that we configured the form with, he was fine with that. It wasn’t really a big code change, and that enabled us to cast that pointer to the structure of the entire field definition because it was the first field in the structure, and this was C programming, and that is what the byte code looked like. It always aligned on a word boundary.
Getting Creighton to update the library so that we could do character by character validation, or having a pop-up style field when there were a limited number of selections just wasn’t in his game plan, but he was a pleasant member of the team and he helped us greatly. Even if we could re-design the Expert Systems library to be more interactive and re-entrant, we didn’t have an interface that supported the necessary structure. So we punted on that and did the best we could with screen organization. Over time, I took responsibility for the library, and I built in some new features that allowed us to go as far with this type of interface as we could go. I hunted through the tabs code and documentation and found that there were feature we could and eventually did utilize.
Robin came up with structure for saving the data, and the routine for formatting the output into the CLP that John and Harry had designed. He wrote this pretty fast, and we had samples to verify, and at the time we proposed that the data simply be a file format per order. So, save it to a file, pass that file around to various people to add their parts to the process, and then validate it, print it, and attach it to the order. We had a working prototype, because that was the nature of what we did. It wasn’t a mock up, but a working prototype, and we sold the concept as something that we could turn issues around quickly with. In those days, big embedded systems required months of process and only delivered 1 or 2 releases a year, whereas we were proposing nightly updates if needed. Everyone who was filling out these paper forms was anxious to get their hands on our software.
Our first installation was in Chicago, and Robin, and I flew out along with Yat Lee. Chicago had a 3B2-300 lying around unused, and we could install on that. They had a printer hooked up, and a few terminal devices, and they would give it a go. The managers name was Ron, and his last name escapes me, but he was very receptive, and he had a small team of people that were responsible for the data, and we spent time talking to them, and then we presented our system with some training materials, and we let them play with it. We took notes, so we could work on updates when we returned to NJ, and basically the installation went well. John and Harry were there as well, and when end of the day arrived, and we started to mull over dinner, Ron suggested a German Restaurant (name forgotten) that was in his family. It sounded decent, and we took a cab over. Ron had called ahead so they would be looking for us, and we were well taken care of when we arrived. The place was packed, but we managed to a couple of small tables in the bar area where we set ourselves up. We ordered beer, and when it came to food, we saw that they had a giant appetizer of pretty much everything. We ordered a couple of them, and spent the next few hours drinking German lagers, and ordering 2 more full appetizer samplers. Prominent in each sampler was the steak tartar, which was absolutely delicious. Robin took his first bite of that, looked at me with wide eyes, and smiled. “If you haven’t tried this yet, then dig in!”. Robin like food. Though I had a bad history with steak tartar that dated to a dinner at my grandmothers when I was just a teen (we kids made multiple trips to the bathroom to spit it out), I sampled it and was blown away by how fresh and delicious it was.
We returned to that restaurant every business trip we managed to secure in Chicago, and we had the same thing every time. Beer and the appetizer sampler.
By the middle of the next year we had made trips to Pleasanton Ca, Atlanta Ga, Denver Co, and White Plains, NY to install our latest releases, and provide training. Each site employed a handful of people to do the data entry, and another handful of “validators” to make sure the order was accurate before it went out. Even though we were “validating” the data, this was AT&T, and for the most part, there was a feeling that there are people in the process who know more than any damn software, and we should develop a process that included utilizing their skill set to look over and review the data. If we said it passed, and they said it passed, then the data was marked validated, and the ordering person could print the output and attach to their order.
Let’s talk a little bit more about this period. In 1982 AT&T was found to be a monopoly, and was ordered to divest itself of its local operations and maintain its hold on long distance. This fostered a new market for communications where the regional bell operating companies would stick to local for some time, but other companies could enter and compete for long distance, long haul business. AT&T was the big player, and their hands were tied to make it possible for others to enter the market. What happened, was many things. After years of a stagnation, the economy finally started to get some traction and took off. Needed were lots of new communications services, and there were plenty of providers out there that wanted in on AT&T’s business. Many of these companies started from nothing, and having nothing, could build new processes pretty freely because they had no embedded systems. As a result they could do certain things “faster” than AT&T. Private line provisioning at AT&T was something like 60 business days per location, and that was just too much time. Think about it today, you go into a mobile center, buy a phone, and at the counter your phone is provisioned to work. Right there, right then it works, and you walk out with a working phone. That was unheard of in those days. For most private line there was some coordination between the local companies and AT&T, but really the brunt of the time was simply in the process followed by years of entrenched systems where data was “copied” from system to system, and every time data got copied, it got copied wrong. These errors added time and re-work to the process.
The net result of this was there was more and more demand for services. AT&T went through process improvement after process improvement, and eventually the notion of using “switched” services became the golden goose. Why build out private lines to everyone, when they already had lines to their local office. What if when they picked up their phone, and dialed “1” for example, you knew it was long distance, so switch them to AT&T at that point. If you could do it for “1” you could do it for other service configurations as well, and so “switched” services were born, and the notion of “PIC”ing your service provider was invented, and this also drove demand. PICing was a process of sending a series of billing and working telephone numbers to the regional Bell companies, and for a nice fee of $5 per number they would PIC those lines so that they always routed to AT&T services. There were millions of these transactions, and worse, competition. Recall a story about Wells Fargo sales folks signing their clients up for unwanted services? Well, MCI used to issue PIC orders against AT&T customers to “steal” the service. It was a nightmare.
The CLP grew into Customer Data Information System (CDIS) and we brought on more people to work the project. Switched SDN was a big seller, and it required a lot of data collection and coordination. In those early days, we were still handling system issues directly from our desk phones, but we worked M-F 9-5 or 6 (really we worked all the time), but we had users in California, and with the time difference we needed to cover them. Also because of demand, sometimes they were working longer hours, so we had hours at both ends of the day to be available. That was when Robin got the brilliant idea that we would get a single Mobile Phone unit and whoever was on call would carry that. When an issue came in, they would take the issue, and try to rally a team to solve the problem. Mobile phones in those days were big boxes with a real phone handle, and they only worked in so many places. Usually urban, and some suburban locations, and Monmouth County was one such suburban location. AT&T was all over the middle to northern state. We had coverage because this were a lot of service originated.
Robin calls me into his office to fill out the application, and we were going to list both of ourselves as the main users. After the usual level of information, the application wanted our birthdays, so since Robin was filling in my information he asked me my birthday and I replied “March 27, 1959”. Robin put the pen down, and looked up at me and said “bullshit!”. I replied that I was not bullshitting him (which of course I could have), and that I had given him my correct birthday. He asked to see my driver’s license as proof, and since I wasn’t lying to him, I produced it, and his jaw dropped when he saw the date. He removed his own driver’s license and gave it to me. Besides his goofy photo id, it said March 27, 1951, and so we found out that day that we shared the same birthday.
As said earlier, the project was growing, and with switched services, on board came Sandy Corr as our Corporate representative, and requirements coordinator. On development we expanded immensely. By this time the project had grown to the point where Robin was promoted to Supervisor of development, and we had a supporting team of System Engineers, and some testers. Testers were still under Robins group, but that would change later.
Cheng-Hui Lin came on board, along with many others including Richard Yeh, CY Tong, Wing Choi, Shiou-Ming Hsieh. We had long moved to a database, and we had started to integrate the separate regions together. Mostly the data was regional, but there was some advantages for large customers who had locations in many regions, to have that data available when managing new orders, and for that reason, the system moved from being located regionally to being centrally located, but regionally divided, and central to that was a common system to which national data was resolved.
As we moved into the 90’s, PCs were getting better, and faster, and the interfaces on them were improving. Also improving was the user experience, and to that effect, event driven interfaces were the future. What wasn’t known then was that the Web was around the corner, and that PC’s would win the PC vs workstation war based simply on economics. Moore’s law kept raising the computing power on PCs and eventually laptops, and workstations were just too cumbersome, and Unix wasn’t a homeowner’s personal OS of choice. However until that battle was lost, CDIS launched a plan to build a desktop interface on a workstation, and Phillip Tsen was brought in for that. When Robin introduced us the first poignant point was that we all shared the same birthday! So now, we were one more in the same group with the same birthday. My memory about the timeline is a little vague, but I think Ammie Chou was already in the group, and during some meeting when the subject came up, she informed us that she too had March 27th as her birthday, and so in a single medium group in AT&T there were four of us with the same birthday. In those years we made it a point to go out for lunch together, and for many years after that, where ever we were, we took a day to celebrate.
That continue well into the early oughts, and then though it died out as physical encounter, we maintained at least an email addressing the day for a few more years. It wasn’t until Social Media that we started to find each other again. Phillip was soon to retire from IBM when he and I hooked up again. Robin found his way onto Facebook, and we kept track of each other. Among other things, another common interest between Robin and myself was photography. I came from a family background of retail photography, and Robin was simply interested in it as a hobby, and we used that common interest to spend non-work hours together meeting for photo shoots, as well as taking darkroom classes at Brookdale together. Though we did mostly 35mm, we also spent time doing medium and large format shoots. At times we gathered during lunch to present slide shows of our work where our co-workers could come and catch some shuteye.
Editing this, I recall another classic Robin story. I am not sure how to preface this. When you meet someone who had lived through the Great Depression, you usually find that person to be quite frugal, even well into the years in which they really aren’t wanting for anything. Same holds true for refugees, or immigrants who came to this country with nothing and built something. They just won’t waste a nickel. How does this relate to Robin? Well, I can’t really say what hardships Robin ever experienced. He was an only child, and as far as I can tell, his Mother made sure he had lots of opportunity. That all said, Robin invested well, and reaped rewards, but when it came to saving even the smallest change, he was all in. Case in point. He schedule a vacation for his family and to save money, he asked me to drive him to the airport so he wouldn’t have to leave a car in the long term lot. I said, “Sure!” I drove to his house, and drove them to the airport in their car, however, Robin wouldn’t let me take the Parkway, or Turnpike because he didn’t want to pay tolls. I took those roads upon my return.
Fast forward to his return. I was sick as the sickest dog. Under the weather, behind the eight ball, under a rock, not myself, feverish and home from work because I was in shit condition, but Robin and his family were due in, and I was supposed to pick them up. So I didn’t even change out of my pajamas. I drove over to his house, grabbed his vehicle and jumped on the parkway. At the first toll plaza I realized I didn’t bring a dime, and this was pre-ez-pass, so I pulled up to a cashier and told him I was sorry, but I jumped in the car with nothing. He kindly threw a quarter in the bin, gave me a tag and said “Next time you come through, hand this and a quarter to the taker, and I will be good.”. I jumped off the parkway, and took 1,9 to the Airport thinking, I’ll pick up Robin, we’ll come back and hit the toll plaza (again, prior to today’s one way toll system), and I’ll make good on that toll takers kindness. I get there, and find Robin and his family, and we start to head home. Turns out they were traveling out of the country, and no-one had any American money, so it was back to Holmdel along the free roads.
Two weeks later, I made a special trip to the Raritan Toll Plaza just to repay the toll worker’s kindness that night.
Ammie reached out to me a few weeks back and told me Robin has been back in the area for almost a year, and we wanted to get together for a March 27th lunch this year. He knew I was still around, and Phillip had retired from IBM was around as well. She called me the day prior to my leaving on vacation with my family, but I promised to get back in touch when I returned. I did think it was odd that Robin had been back for a year and hadn’t reached out, but I was on vacation and that soon was a distant memory. A few days before returning home I got a text from Ammie on whether I was home yet or still on vacation, and though I was confused about why Ammie was texting me (I thought it was my friends daughter Ammie , same spelling kind of), I figured I would check back with her when I got home. That Monday night when I returned my phone rang with a number not recognized, and as soon as I heard Ammie Chao’s voice I realized my mistake, and I knew something was up. Our birthday was more than a month away, and the urgency told me that something wasn’t right. Robin had been diagnosed with stage 4 pancreatic cancer, and that was a year ago. The treatments probably bought him time, but his time was running out.
The next day Ammie and Phillip helped Robin get to the hospital where he spent time in the emergency room until they admitted him into the Jane Booker cancer wing. I came down after work Tuesday and met them and Robin there. I hadn’t seen Robin in probably fifteen years, and so my memory of him was the big healthy looking, great appetite Robin Ying of old, and until I looked into his eyes, I wasn’t sure I was looking at the same person. Just like Donald Wormley (also of CDIS days and later I worked international work with him), Don was a food loving great man whom cancer had gotten a hold of and brought down to the point where I didn’t recognize the man any longer. Robin wasn’t that bad, but the cancer had and was still pretty bad. He was alert, and he managed a great smile for a group March 27th birthday photo.
The next day Robin was transferred to the ICU. Joanne Penn (recruited as Joanne Louie in the early days) managed to get down there on Thursday, and texted me that I should make another visit. Robin was in pretty sad shape, and was probably going to die soon. Since he was in ICU, there were specific hours of visitation so after feeding my family, I made my way over there for the 8:30 visiting hours. When I got there he was by himself in his room and he was sleeping, and there was no way I was going in there and disturbing him. He was hooked up to many more things, and he looked much worse than only two days prior. I chose to go wait in the waiting room for some time and then check back before leaving. This time I found a group of friends from his church there in prayer. I respectfully stayed out until they finished, and then I entered. Robin was awake, but was really struggling. One of the church friends recognized me and mentioned it to Robin, but he seemed very incoherent at that point. My last memory of him was struggling to breath, and I knew right then that was the last time I would see my old friend alive.
Robin passed away early Saturday morning with his family at his side. I attended a service for him yesterday at the Monmouth Chinese Christian Church in Middletown where we all paid our last respects, and we met up with old friends and colleagues. Many familiar faces, and not simply from my CLP/CDIS days! Hahn Tee from my Tellium days, and Jason from my Lucent/Alcatel days attended the same church as Robin, and recognized me, or at least my name was recognized, and they both came over to greet me. Many faces from CDIS though, and we all felt a special kinship from that project. While there may have been trials and tribulations, we all really got along pretty well, and enjoyed working together. It may be cliche to say we were like a family, but aside from family, we spent a LOT of time with each other back in the day. We probably saw more of each other than our families in those days. It was the job that brought us together, so without the job, we of course all went our separate ways, running into each other every now and then. I may have been more recognizable because of my height, and I was one of the few non-Asian people working that project, but everyone was familiar, and we all held positive perspectives on those times.
We all learned a little more about our friend and former colleague yesterday. He had a pretty rich life, and he lived it very well, and he leaves behind a son and daughter who were raised well, and who unfortunately have to proceed forward without their father to call upon. He gave them great values, and terrific opportunities, and they will feel his touch the rest of their lives.
Beautiful, Eric. Thank you so much. We really were a family. I’ve never had a job before or since that I loved as much as working with CDIS.
Thank you for sharing this wonderful history/memory.
You are welcome