Almost exactly four years since its inception, ROC Engineering Change Proposals (ECP) 0289 "Provide Digital Communications between OPUP and RPG" and ECP 0374 "OPUP Refresh" is coming to fruition. All told, 254 new discrete DoD NEXRAD Digital Private IP (PIP) Wide Area Network (WAN) service orders will be released to install this new service at all RPGs and all Air Force OPUP locations. Verizon Business (VzB) was competitively awarded this new service in December 2009 via the GSA Networx Universal contract. Telecommunication Service Orders (TSRs) for the new service started being released on July 2nd, 2010 and all services are aggressively targeted for completion.

The following links provides an overview diagram of the planned PIP network and schedule. Additional info specific to the RPG and OPUP ends is provided below.

Network Overview

Schedule

Site-specific dates provided in the schedule are the best available at present and will be updated through the course of the service implementation. VzB is contractually obligated to provide e-mail notifications for all facets of the new service implementation. While VzB is ironing out issues with their notification process, ROC will extract available schedule and implementation information and email it to each affected field sites. This is an interim measure and will cease once VzB provides the required field notices. Maintenance staff for the RPG ends, POCs for the Air Force base comms squadrons, and OPUP managers will receive e-mail copies of the Telecommunications Service Orders (TSRs) from DISA for their respective location(s). Rows 106A and 106B of the TSR shows the contracted service dates for the new network capability. The schedule link will be updated over the course of the deployment to reflect updates for each aspect of the service activation – from T1 install by the LEC to equipment install and service activation by VzB. ROC has recommended that those on the e-mail distribution for the TSRs also receive e-mail notices from VzB for each phase of the implementation.

The new PIP WAN will include Managed Network Services (MNS) from VzB and numerous built-in support mechanisms for NEXRAD customers. For example, as each circuit is activated, the cognizant site POCs, which are generally the same POCs that are recipients of the TSRs, will be “registered” for VzB’s Networx Web portal. Registration will ensure the receipt of auto-notifications when a trouble ticket is opened on the service including tickets opened proactively by VzB under the MNS tier of services due to an alarm on the circuit) and tickets for Scheduled Maintenance Notices that could impact the PIP WAN. Radar Focal Points for DoD, FAA, and NWS RPGs will see notifications relative to the circuits on the RPG end(s). OPUP managers will be registered for circuits on both the OPUP and RPG ends, a capability brand new to OPUP telecoms. The same “entitlements” includes capability to monitor the applicable circuits via the Networx portal and allows the interested end users to follow VzB’s progress resolving any trouble tickets via the web portal.

The following links provide the specifics from related system modifications to the planned rollout on a system-specific basis:

RPGs	Small OPUPs	Large/Medium OPUPs

Like other recent large scale NEXRAD telecom projects, the ROC will support each site through completion of deployment. Please call the NEXRAD Hotline if you have any questions concerning these projects and their respective field deployments.

RPGs - DoD OPUP-NEXRAD PIP Network

The planned RPG configurations to support the new PIP network can be viewed at the following links:

Typical NWS WFO or DoD WSR-88D Shelter	NWS WFO with 1 RPG & Hosting DoD MSCF

NWS WFO with 2 RPGs	FAA WSR-88D Shelter

In some respects, this change is similar to the FAA FTI IP implementation (supported via Mod Notes 96 and then 101). Like the FAA transition to PIP WAN, the changes necessary to support the OPUP IP transition have been incrementally added to RPG narrowband adaptation going back several Builds and will continue through RPG Build 12. Current RPG Builds support all existing OPUP PIP connectivity requirements.

A brief chronology of the recent hardware changes that got us here ...

Mod Note 125/TCTO 728/FAA EEM CHG 113 Chap 105, Issuance Date 27 July 2009, was applicable to NWS locations with Distant MSCF for DoD radars, NWS offices with 2 RPGs, and all DoD and FAA RPGs. This modification note was intended to be completed prior to RPG router refresh (Mod Note 116).

For NWS, the Mod added an 8-port LAN switch (A38) to be located between VzB's router and the RPG I/O panel. The affected NWS locations were given a 50 blue "straight-through" W113D (and W113E for locations w/two RPGs) cable to extend between the A38 LAN switch and I/O panel CP1.

DoD locations were given a 10' blue "straight-through" 70/170W200 cable to install between the RPG router and the inside of the I/O panel to CP1. A 50' red "cross-over" cable - W113 – was provided to extend between the outside of the RPG I/O Panel (P1 to CP1) and the other end was to be left coiled in proximity to where you want the Provider to install their router.

FAA locations received an 8-port LAN switch to replace the old (A32) 5-port LAN switch and a new 8-port LAN switch (A36) for OPUP Digital. The mod installed cables W113A and W113B (50-foot blue straight-through) from the LAN Switch (A36) to the respective RPG channels I/O panels - P1 to CP1. The mod also re-labeled cable 70/170W233 to 70/170W200 and moved the I/O panel end only it from CP9 to CP1.

Mod Note 133/TCTO 747/FAA EEM CHG 142 Chap 134 completes the related hardware modifications for OPUP digital on the RPG end. Site technicians are to install/connect the appropriate cable from either the 8 port LAN Switch (A38 or A36) installed per Mod Note 125 or from CP1 to the Telco router.

As part of the PIP service turn-up, NEXRAD site technicians will need to make a POTS dial line available to the VzB technician so they can connect it to a VzB provided out of band (OOB) modem. VzB’s OOB modem is part of the Managed Network Services (MNS) component. Once the service is activated, the POTS line can be returned to its former use. Ideally NEXRAD technicians will take measures to ensure the POTS line can easily and quickly be connected to the OOB modem in the event an issue arises where the VzB loses in-band access and needs the OOB capability to troubleshoot the router. Other than during the service turn-up, this measure should rarely be needed. VzB has informed us that normal troubleshooting of the T1 will not be put on-hold in absence of a POTS line to the OOB modem.

The ROC will monitor all service activations. As service is activated, the site technicians will complete cross-connect from the RPG (or 8 port LAN switch, if applicable) to the VzB router per Mod Note 133. The ROC will then use a Digital OPUP to verify product receipt from each RPG and complete acceptance of the service. The immediate cross-connect is important, so that we leverage off of the VzB team still being engaged and the VzB tech still being on-site to help troubleshoot if problems occur with service acceptance. This sequence will be reemphasized near the date for service turn-up.

The "View By Line" option of the WSR-88D comms documentation for all RPGs will reflect Lines 1 -9 being associated with the new PIP service once the circuit info is made available by the VzB. Product Distribution Comms Status (PDCS) screen Line numbers 1 and 2 are new DoD WAN OTR ports that are analogous to AWIPS WAN OTR ports on most RPGs. DoD WAN OTR Lines 1 and 2 are not active until RPG Build 12, and will not be used operationally until OPUP Software Release 12.1 is installed on the OPUP ends. Once all the associated OPUP(s) are successfully transitioned to IP for a given RPG, ROC will remove information relating to OPUP analog point-to-point telecoms from the RPG comms documentation. ROC will separately track disconnects of the OPUP analog telecoms with AFWA and DISA. For CONUS RPGs, Sprint will be requested to remove their T1 channel bank hardware, when appropriate, from the WFO comms rooms and DoD radar shelters.

The Marine Corps will not be participating in the OPUP upgrade. The telecoms for these systems will phased out by the Navy/Marine Corps and the ROC will try to obtain a disconnect schedule and provide it to radar focal points at all applicable RPG sites.

There will only be one DoD physical T1 access circuit regardless of the number of OPUP associated user connections. Individual OPUPs are assigned a unique TCP port on each RPG. The RPG software site specific data CD sets a port speed flow control of 128, 256, 384, 1500 Kbps to match the contract purchased port speed for a given location. If there are two RPGs at a WFO, the RPGs will share the physical T1 access and split the purchased port speed according to the number of OPUP connections that were planned for each RPG.

Small OPUPs will migrate to the PIP WAN communications via Mod Note 119. The overall migration is combined with a complete operating system and hardware technical refresh for OPUP Build 12. Small OPUPs have to install a new router as part of the tech refresh and the new router does not support analog communications. There is no “parallel operations” requirement to keep analog comms as a backup, so the small OPUPs will be “hot cut” as soon as their refresh is complete. The VzB PIP services must be ready at the OPUP and its respective RPG(s) prior to the OPUP manager’s completion of the Tech Refresh/Build 12.0 install (Mod Not 119).

Air Force Operational Weather Squadron (OWS) Large OPUPs will establish their required dedicated connection to each RPG and this will disable the analog connection to the same RPG.

DoD Distant MSCFs (DMSCF) will be migrated to the new DoD NEXRAD PIP network in a future software release.

Small OPUPs - DoD OPUP-NEXRAD PIP Network

Mod Note 119\TCTO 31P1-4-108-747 (Draft) will be performed by Small OPUP managers to accomplish the hardware refresh and transition the OPUP to Build 12.0 and the new PIP comms. Once retrofitted, the Small OPUP will no longer support the pre-existing analog telecoms. The pre-existing analog telecoms will be scheduled for disconnect as soon as the Build 12.0 OPUP is on-line and receiving products from its associated RPG.

The Small OPUP refresh will result in a complete new system that supports the VzB PIP telecoms. The refresh will also provide significant improvements in processing, memory, disk capacity. The operating system changes from Solaris to Red Hat Linux Enterprise. The post-Build 12.0 OPUP configurations are depicted with the following links.

• Typical Small OPUP
• Vandenberg AFB Small OPUP
• Lajes AB Small OPUP

Before starting Mod Note 119, the legacy OPUP must be upgraded to Build 11.2 in order to ensure compatibility with RPG Build 12.0, which started field-wide release on 08/26/2010, and to ensure predefined adaptation data – RPS Lists, alert areas, etc., can be merged forward to the Build 12.0 OPUP.

As specified in the small OPUP TSRs for the PIP service, the base comms squadrons are supporting the effort as follows:

• To extend the new service from the base central office to the base weather/operations building.


• To extend an Ethernet cable provided by the ROC per OPUP Mod Note 119 (or Mod Note 133 for Vandenberg and Lajes) from VzB’s router (or ROC provided A38 LAN switch) to the refreshed OPUP.


• Provide a temporary Plain Old Telephone Service (POTS) dial line to the VzB technician so they can connect it to an out of band (OOB) modem that will be part of the VzB’s equipment. See image of the VzB's OOB Multi-Tech MT5634ZBA-Global Multi-Modem. The VzB's OOB capability is part of the Managed Network Services (MNS) component of the new telecoms service. Once the service is activated the POTS line can be returned to its former use.

OPUP managers will need to closely monitor the new PIP telecom activations to:

• Ensure the POTS is extended to the VzB’s OOB modem as noted above. OPUP managers will need to record and report the POTS line phone number to the NEXRAD Hotline activation support team as noted below. The ROC will then add that info to each site’s “profile”. This measure is necessary in the event the POTS line needs to be returned to the VzB's OOB modem in the future. Other than the service turn-up, this should be a rare occasion, but better to be prepared. VzB indicates normal T1 troubleshooting mechanisms will not be put on-hold as a result of the POTS line not already being extended to the OOB modem, but it the VzB’s router were to lose its configuration, say for example from an extended power outage, the POTS line might then need to be returned to the OOB modem. (An in the interim of course the OPUP will have no connectivity.)


• Learn from the service activation how to interpret the LEDs on the VzB's router and identify how the cables are routed and marked (to facilitate troubleshooting should the router lose connectivity or power or if the interconnecting cables fail.) It is recommended that OPUP managers request 24x7 access to the VzB router to perform primary/secondary fault isolations as per the OPUP technical manual.

The ROC will monitor all PIP service activations and perform acceptance on behalf of the government. RPGs will be verified using the ROC Hotline Build 12 small OPUP. On the OPUP ends the ROC will monitor the service activations and will perform common network tests to verify connection and routing to the small OPUP VzB router. Quite some time could elapse between the initial PIP service activation date and the date you activate your new OPUP, so OPUP managers are asked to contact Mark Albertelly or Tanylle Casper at the NEXRAD Hotline to re-test the IP path to both your RPG and the VzB router at the small OPUP before starting the Mod Note 119.

As noted with the TSRs each location is receiving, VzB, “…is responsible for overseeing end-to-end service delivery and activation in the customer facility. [The VzB] will sub-contract with the Local Exchange Carrier (LEC) to extend T1 circuits from the central office to the customer's demarc. The ROC has no visibility into the base comms squadron roles and will rely heavily on OPUP managers at each location to help ensure the new PIP service delivery and activation schedules are met.

As the PIP telecoms is being prepared for activation at each of the Small OPUP locations, the ROC comms documentation Web page for each OPUP (see: https://www.roc.noaa.gov/comms/) will be updated to show the new telecoms circuit data while maintaining info relative to the preexisting analog telecoms. Any OPUP managers that do not have their access info for the OPUP Comms Documentation Web page are asked to phone the NEXRAD Hotline to obtain that info.

Large & Medium OPUPs - DoD OPUP-NEXRAD PIP Network

AFWA Large and Medium OPUPs will receive a tech refresh, Build 12.0, and the digital communications enhancement as a single ROC installed retrofit. The large and medium OPUPs will support both analog dial and digital communications until all the radars are migrated to digital communications. Resulting OWS configuration examples are provided with the following links:

15th OWS & 25th OWS	26th OWS	17th OWS

As specified in the OWS TSRs for the PIP service, the base comms squadrons are supporting the effort as follows:

• To extend the new service from the base central office to the base weather/operations building.


• To extend an Ethernet cable provided by the ROC from the VzB router (or ROC provided A38 LAN switch) to the refreshed OPUP.


• Provide a temporary Plain Old Telephone Service (POTS) dial line to the VzB technician so they can connect it to an out of band (OOB) modem that will be part of the VzB’s equipment. See image of the VzB's OOB Multi-Tech MT5634ZBA-Global Multi-Modem. The VzB's OOB capability is part of the Managed Network Services (MNS) component of the new telecoms service. Once the service is activated the POTS line can be returned to its former use.

OPUP managers will need to closely monitor the new PIP telecom activations to:

• Ensure the POTS is extended to the VzB’s OOB modem as noted above. OPUP managers will need to record and report the POTS line phone number to the NEXRAD Hotline activation support team as noted below. The ROC will then add that info to each site’s “profile”. This measure is necessary in the event the POTS line needs to be returned to the VzB's OOB modem in the future. Other than the service turn-up, this should be a rare occasion, but better to be prepared. VzB indicates normal T1 troubleshooting mechanisms will not be put on-hold as a result of the POTS line not already being extended to the OOB modem, but it the VzB’s router were to lose its configuration, say for example from an extended power outage, the POTS line might then need to be returned to the OOB modem. (An in the interim of course the OPUP will have no connectivity.)


• Learn from the service activation how to interpret the LEDs on the VzB's router and identify how the cables are routed and marked (to facilitate troubleshooting should the router lose connectivity or power or if the interconnecting cables fail.) It is recommended that OPUP managers request 24x7 access to the VzB router to perform primary/secondary fault isolations as per the OPUP technical manual.

Build 12 for Large OPUPs will only replicate existing RPG “dedicated” connectivity. A Build 12.1 point OPUP software update will be released to replace analog dial with a WAN one time request (WAN OTR) capability for additional on-demand product access to non-associated RPGs. Small OPUPs will also receive the 12.1 WAN OTR capability but their systems will be configured to connect to a separate port on each RPG from the port reserved for OWS WAN OTR access.

As the new PIP telecoms is being prepared for activation at each of the OWS locations, the ROC comms documentation Web page for each Large and Medium OPUP (see: https://www.roc.noaa.gov/comms/) will be updated to show the new telecoms circuit data while maintaining info relative to the preexisting analog telecoms. Any OPUP managers that do not have their access info for the OPUP Comms Documentation Web page are asked to phone the NEXRAD Hotline to obtain the necessary Web page access info.